Unverified Commit 853d0a3f authored by Maximilian Bosch's avatar Maximilian Bosch Committed by GitHub
Browse files

Merge pull request #199150 from Ma27/grafana-fixup 

nixos/grafana: documentation/warning improvements after #191768
parents bd3a8c7b 4a73fad5

nixos/doc/manual/from_md/release-notes/rl-2211.section.xml

+138 −14
Original line number Diff line number Diff line
@@ -1206,22 +1206,146 @@ services.github-runner.serviceOverrides.SupplementaryGroups = [ 
      </listitem>

      <listitem>

        <para>

          The <literal>services.grafana</literal> options were converted

          to a

          The module <literal>services.grafana</literal> was refactored

          to be compliant with

          <link xlink:href="https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md">RFC

          0042</link> configuration.

          0042</link>. To be precise, this means that the following

          things have changed:

        </para>

        <itemizedlist>

          <listitem>

            <para>

              The newly introduced option

              <xref linkend="opt-services.grafana.settings" /> is an

              attribute-set that will be converted into Grafana’s INI

              format. This means that the configuration from

              <link xlink:href="https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/">Grafana’s

              configuration reference</link> can be directly written as

              attribute-set in Nix within this option.

            </para>

          </listitem>

          <listitem>

            <para>

          The <literal>services.grafana.provision.datasources</literal>

          and <literal>services.grafana.provision.dashboards</literal>

          options were converted to a

          <link xlink:href="https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md">RFC

          0042</link> configuration. They also now support specifying

          the provisioning YAML file with <literal>path</literal>

          option.

              The option

              <literal>services.grafana.extraOptions</literal> has been

              removed. This option was an association of environment

              variables for Grafana. If you had an expression like

            </para>

            <programlisting language="bash">

{

  services.grafana.extraOptions.SECURITY_ADMIN_USER = &quot;foobar&quot;;

}

</programlisting>

            <para>

              your Grafana instance was running with

              <literal>GF_SECURITY_ADMIN_USER=foobar</literal> in its

              environment.

            </para>

            <para>

              For the migration, it is recommended to turn it into the

              INI format, i.e. to declare

            </para>

            <programlisting language="bash">

{

  services.grafana.settings.security.admin_user = &quot;foobar&quot;;

}

</programlisting>

            <para>

              instead.

            </para>

            <para>

              The keys in

              <literal>services.grafana.extraOptions</literal> have the

              format

              <literal>&lt;INI section name&gt;_&lt;Key Name&gt;</literal>.

              Further details are outlined in the

              <link xlink:href="https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#override-configuration-with-environment-variables">configuration

              reference</link>.

            </para>

            <para>

              Alternatively you can also set all your values from

              <literal>extraOptions</literal> to

              <literal>systemd.services.grafana.environment</literal>,

              make sure you don’t forget to add the

              <literal>GF_</literal> prefix though!

            </para>

          </listitem>

          <listitem>

            <para>

              Previously, the options

              <xref linkend="opt-services.grafana.provision.datasources" />

              and

              <xref linkend="opt-services.grafana.provision.dashboards" />

              expected lists of datasources or dashboards for the

              <link xlink:href="https://grafana.com/docs/grafana/latest/administration/provisioning/">declarative

              provisioning</link>.

            </para>

            <para>

              To declare lists of

            </para>

            <itemizedlist spacing="compact">

              <listitem>

                <para>

                  <emphasis role="strong">datasources</emphasis>, please

                  rename your declarations to

                  <xref linkend="opt-services.grafana.provision.datasources.settings.datasources" />.

                </para>

              </listitem>

              <listitem>

                <para>

                  <emphasis role="strong">dashboards</emphasis>, please

                  rename your declarations to

                  <xref linkend="opt-services.grafana.provision.dashboards.settings.providers" />.

                </para>

              </listitem>

            </itemizedlist>

            <para>

              This change was made to support more features for that:

            </para>

            <itemizedlist>

              <listitem>

                <para>

                  It’s possible to declare the

                  <literal>apiVersion</literal> of your dashboards and

                  datasources by

                  <xref linkend="opt-services.grafana.provision.datasources.settings.apiVersion" />

                  (or

                  <xref linkend="opt-services.grafana.provision.dashboards.settings.apiVersion" />).

                </para>

              </listitem>

              <listitem>

                <para>

                  Instead of declaring datasources and dashboards in

                  pure Nix, it’s also possible to specify configuration

                  files (or directories) with YAML instead using

                  <xref linkend="opt-services.grafana.provision.datasources.path" />

                  (or

                  <xref linkend="opt-services.grafana.provision.dashboards.path" />.

                  This is useful when having provisioning files from

                  non-NixOS Grafana instances that you also want to

                  deploy to NixOS.

                </para>

                <para>

                  <emphasis role="strong">Note:</emphasis> secrets from

                  these files will be leaked into the store unless you

                  use a

                  <link xlink:href="https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#file-provider"><emphasis role="strong">file</emphasis>-provider

                  or env-var</link> for secrets!

                </para>

              </listitem>

              <listitem>

                <para>

                  <xref linkend="opt-services.grafana.provision.notifiers" />

                  is not affected by this change because this feature is

                  deprecated by Grafana and will probably removed in

                  Grafana 10. It’s recommended to use

                  <literal>services.grafana.provision.alerting.contactPoints</literal>

                  instead.

                </para>

              </listitem>

            </itemizedlist>

          </listitem>

        </itemizedlist>

      </listitem>

      <listitem>

        <para>

nixos/doc/manual/release-notes/rl-2211.section.md

+60 −3
Original line number Diff line number Diff line
@@ -379,9 +379,66 @@ Available as [services.patroni](options.html#opt-services.patroni.enable). 


- The `services.matrix-synapse` systemd unit has been hardened.



- The `services.grafana` options were converted to a [RFC 0042](https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md) configuration.

- The module `services.grafana` was refactored to be compliant with [RFC 0042](https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md). To be precise, this means that the following things have changed:

  - The newly introduced option [](#opt-services.grafana.settings) is an attribute-set that

    will be converted into Grafana's INI format. This means that the configuration from

    [Grafana's configuration reference](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/)

    can be directly written as attribute-set in Nix within this option.

  - The option `services.grafana.extraOptions` has been removed. This option was an association

    of environment variables for Grafana. If you had an expression like



    ```nix

    {

      services.grafana.extraOptions.SECURITY_ADMIN_USER = "foobar";

    }

    ```



    your Grafana instance was running with `GF_SECURITY_ADMIN_USER=foobar` in its environment.



    For the migration, it is recommended to turn it into the INI format, i.e.

    to declare



    ```nix

    {

      services.grafana.settings.security.admin_user = "foobar";

    }

    ```



    instead.



    The keys in `services.grafana.extraOptions` have the format `<INI section name>_<Key Name>`.

    Further details are outlined in the [configuration reference](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#override-configuration-with-environment-variables).



    Alternatively you can also set all your values from `extraOptions` to

    `systemd.services.grafana.environment`, make sure you don't forget to add

    the `GF_` prefix though!

  - Previously, the options [](#opt-services.grafana.provision.datasources) and

    [](#opt-services.grafana.provision.dashboards) expected lists of datasources

    or dashboards for the [declarative provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/).



    To declare lists of

    - **datasources**, please rename your declarations to [](#opt-services.grafana.provision.datasources.settings.datasources).

    - **dashboards**, please rename your declarations to [](#opt-services.grafana.provision.dashboards.settings.providers).



    This change was made to support more features for that:



    - It's possible to declare the `apiVersion` of your dashboards and datasources

      by [](#opt-services.grafana.provision.datasources.settings.apiVersion) (or

      [](#opt-services.grafana.provision.dashboards.settings.apiVersion)).



    - Instead of declaring datasources and dashboards in pure Nix, it's also possible

      to specify configuration files (or directories) with YAML instead using

      [](#opt-services.grafana.provision.datasources.path) (or

      [](#opt-services.grafana.provision.dashboards.path). This is useful when having

      provisioning files from non-NixOS Grafana instances that you also want to

      deploy to NixOS.



      __Note:__ secrets from these files will be leaked into the store unless you use a

      [**file**-provider or env-var](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#file-provider) for secrets!



- The `services.grafana.provision.datasources` and `services.grafana.provision.dashboards` options were converted to a [RFC 0042](https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md) configuration. They also now support specifying the provisioning YAML file with `path` option.

    - [](#opt-services.grafana.provision.notifiers) is not affected by this change because

      this feature is deprecated by Grafana and will probably removed in Grafana 10.

      It's recommended to use `services.grafana.provision.alerting.contactPoints` instead.



- The `services.grafana.provision.alerting` option was added. It includes suboptions for every alerting-related objects (with the exception of `notifiers`), which means it's now possible to configure modern Grafana alerting declaratively.

nixos/modules/services/monitoring/grafana.nix

+143 −135

File changed.

Preview size limit exceeded, changes collapsed.

nixos/tests/grafana/provision/default.nix

+63 −41
Original line number Diff line number Diff line
@@ -17,7 +17,7 @@ let 


        security = {

          admin_user = "testadmin";

          admin_password = "snakeoilpwd";

          admin_password = "$__file{${pkgs.writeText "pwd" "snakeoilpwd"}}";

        };

      };

    };
@@ -28,17 +28,24 @@ let 
  };



  extraNodeConfs = {

    provisionOld = {

    provisionLegacyNotifiers = {

      services.grafana.provision = {

        datasources.settings = {

          apiVersion = 1;

          datasources = [{

            name = "Test Datasource";

            type = "testdata";

            access = "proxy";

            uid = "test_datasource";

          }];



        dashboards = [{ options.path = "/var/lib/grafana/dashboards"; }];



        };

        dashboards.settings = {

          apiVersion = 1;

          providers = [{

            name = "default";

            options.path = "/var/lib/grafana/dashboards";

          }];

        };

        notifiers = [{

          uid = "test_notifiers";

          name = "Test Notifiers";
@@ -50,7 +57,6 @@ let 
        }];

      };

    };



    provisionNix = {

      services.grafana.provision = {

        datasources.settings = {
@@ -157,6 +163,22 @@ let 
        };

      };

    };



    provisionYamlDirs = let

      mkdir = p: pkgs.writeTextDir (baseNameOf p) (builtins.readFile p);

    in {

      services.grafana.provision = {

        datasources.path = mkdir ./datasources.yaml;

        dashboards.path = mkdir ./dashboards.yaml;

        alerting = {

          rules.path = mkdir ./rules.yaml;

          contactPoints.path = mkdir ./contact-points.yaml;

          policies.path = mkdir ./policies.yaml;

          templates.path = mkdir ./templates.yaml;

          muteTimings.path = mkdir ./mute-timings.yaml;

        };

      };

    };

  };



  nodes = builtins.mapAttrs (_: val: mkMerge [ val baseGrafanaConf ]) extraNodeConfs;
@@ -172,58 +194,58 @@ in { 
  testScript = ''

    start_all()



    nodeOld = ("Nix (old format)", provisionOld)

    nodeNix = ("Nix (new format)", provisionNix)

    nodeYaml = ("Nix (YAML)", provisionYaml)

    nodeYamlDir = ("Nix (YAML in dirs)", provisionYamlDirs)



    for nodeInfo in [nodeOld, nodeNix, nodeYaml]:

        with subtest(f"Should start provision node: {nodeInfo[0]}"):

            nodeInfo[1].wait_for_unit("grafana.service")

            nodeInfo[1].wait_for_open_port(3000)

    for description, machine in [nodeNix, nodeYaml, nodeYamlDir]:

        with subtest(f"Should start provision node: {description}"):

            machine.wait_for_unit("grafana.service")

            machine.wait_for_open_port(3000)



        with subtest(f"Successful datasource provision with {nodeInfo[0]}"):

            nodeInfo[1].succeed(

        with subtest(f"Successful datasource provision with {description}"):

            machine.succeed(

                "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/datasources/uid/test_datasource | grep Test\ Datasource"

            )



        with subtest(f"Successful dashboard provision with {nodeInfo[0]}"):

            nodeInfo[1].succeed(

        with subtest(f"Successful dashboard provision with {description}"):

            machine.succeed(

                "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/dashboards/uid/test_dashboard | grep Test\ Dashboard"

            )







    with subtest(f"Successful notifiers provision with {nodeOld[0]}"):

        nodeOld[1].succeed(

            "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/alert-notifications/uid/test_notifiers | grep Test\ Notifiers"

        )







    for nodeInfo in [nodeNix, nodeYaml]:

        with subtest(f"Successful rule provision with {nodeInfo[0]}"):

            nodeInfo[1].succeed(

        with subtest(f"Successful rule provision with {description}"):

            machine.succeed(

                "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/v1/provisioning/alert-rules/test_rule | grep Test\ Rule"

            )



        with subtest(f"Successful contact point provision with {nodeInfo[0]}"):

            nodeInfo[1].succeed(

        with subtest(f"Successful contact point provision with {description}"):

            machine.succeed(

                "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/v1/provisioning/contact-points | grep Test\ Contact\ Point"

            )



        with subtest(f"Successful policy provision with {nodeInfo[0]}"):

            nodeInfo[1].succeed(

        with subtest(f"Successful policy provision with {description}"):

            machine.succeed(

                "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/v1/provisioning/policies | grep Test\ Contact\ Point"

            )



        with subtest(f"Successful template provision with {nodeInfo[0]}"):

            nodeInfo[1].succeed(

        with subtest(f"Successful template provision with {description}"):

            machine.succeed(

                "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/v1/provisioning/templates | grep Test\ Template"

            )



        with subtest("Successful mute timings provision with {nodeInfo[0]}"):

            nodeInfo[1].succeed(

        with subtest("Successful mute timings provision with {description}"):

            machine.succeed(

                "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/v1/provisioning/mute-timings | grep Test\ Mute\ Timing"

            )



    with subtest("Successful notifiers provision"):

        provisionLegacyNotifiers.wait_for_unit("grafana.service")

        provisionLegacyNotifiers.wait_for_open_port(3000)

        print(provisionLegacyNotifiers.succeed(

            "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/alert-notifications/uid/test_notifiers"

        ))

        provisionLegacyNotifiers.succeed(

            "curl -sSfN -u testadmin:snakeoilpwd http://127.0.0.1:3000/api/alert-notifications/uid/test_notifiers | grep Test\ Notifiers"

        )

  '';

})