Merge pull request #272380 from chayleaf/gradle2

# Gradle {#gradle}

Gradle is a popular build tool for Java/Kotlin. Gradle itself doesn't

currently provide tools to make dependency resolution reproducible, so

nixpkgs has a proxy designed for intercepting Gradle web requests to

record dependencies so they can be restored in a reproducible fashion.

## Building a Gradle package {#building-a-gradle-package}

Here's how a typical derivation will look like:

```nix

stdenv.mkDerivation (finalAttrs: {

  pname = "pdftk";

  version = "3.3.3";

  src = fetchFromGitLab {

    owner = "pdftk-java";

    repo = "pdftk";

    rev = "v${finalAttrs.version}";

    hash = "sha256-ciKotTHSEcITfQYKFZ6sY2LZnXGChBJy0+eno8B3YHY=";

  };

  nativeBuildInputs = [ gradle ];

  # if the package has dependencies, mitmCache must be set

  mitmCache = gradle.fetchDeps {

    inherit (finalAttrs) pname;

    data = ./deps.json;

  };

  # this is required for using mitm-cache on Darwin

  __darwinAllowLocalNetworking = true;

  gradleFlags = [ "-Dfile.encoding=utf-8" ];

  # defaults to "assemble"

  gradleBuildTask = "shadowJar";

  # will run the gradleCheckTask (defaults to "test")

  doCheck = true;

  installPhase = ''

    mkdir -p $out/{bin,share/pdftk}

    cp build/libs/pdftk-all.jar $out/share/pdftk

    makeWrapper ${jre}/bin/java $out/bin/pdftk \

      --add-flags "-jar $out/share/pdftk/pdftk-all.jar"

    cp ${finalAttrs.src}/pdftk.1 $out/share/man/man1

  '';

  meta.sourceProvenance = with lib.sourceTypes; [

    fromSource

    binaryBytecode # mitm cache

  ];

})

```

To update (or initialize) dependencies, run the update script via

something like `$(nix-build -A <pname>.mitmCache.updateScript)`

(`nix-build` builds the `updateScript`, `$(...)` runs the script at the

path printed by `nix-build`).

If your package can't be evaluated using a simple `pkgs.<pname>`

expression (for example, if your package isn't located in nixpkgs, or if

you want to override some of its attributes), you will usually have to

pass `pkg` instead of `pname` to `gradle.fetchDeps`. There are two ways

of doing it.

The first is to add the derivation arguments required for getting the

package. Using the pdftk example above:

```nix

{ lib

, stdenv

# ...

, pdftk

}:

stdenv.mkDerivation (finalAttrs: {

  # ...

  mitmCache = gradle.fetchDeps {

    pkg = pdftk;

    data = ./deps.json;

  };

})

```

This allows you to `override` any arguments of the `pkg` used for

the update script (for example, `pkg = pdftk.override { enableSomeFlag =

true };`), so this is the preferred way.

The second is to create a `let` binding for the package, like this:

```nix

let self = stdenv.mkDerivation {

  # ...

  mitmCache = gradle.fetchDeps {

    pkg = self;

    data = ./deps.json;

  };

}; in self

```

This is useful if you can't easily pass the derivation as its own

argument, or if your `mkDerivation` call is responsible for building

multiple packages.

In the former case, the update script will stay the same even if the

derivation is called with different arguments. In the latter case, the

update script will change depending on the derivation arguments. It's up

to you to decide which one would work best for your derivation.

## Update Script {#gradle-update-script}

The update script does the following:

- Build the derivation's source via `pkgs.srcOnly`

- Enter a `nix-shell` for the derivation in a `bwrap` sandbox (the

  sandbox is only used on Linux)

- Set the `IN_GRADLE_UPDATE_DEPS` environment variable to `1`

- Run the derivation's `unpackPhase`, `patchPhase`, `configurePhase`

- Run the derivation's `gradleUpdateScript` (the Gradle setup hook sets

  a default value for it, which runs `preBuild`, `preGradleUpdate`

  hooks, fetches the dependencies using `gradleUpdateTask`, and finally

  runs the `postGradleUpdate` hook)

- Finally, store all of the fetched files' hashes in the lockfile. They

  may be `.jar`/`.pom` files from Maven repositories, or they may be

  files otherwise used for building the package.

`fetchDeps` takes the following arguments:

- `attrPath` - the path to the package in nixpkgs (for example,

  `"javaPackages.openjfx22"`). Used for update script metadata.

- `pname` - an alias for `attrPath` for convenience. This is what you

  will generally use instead of `pkg` or `attrPath`.

- `pkg` - the package to be used for fetching the dependencies. Defaults

  to `getAttrFromPath (splitString "." attrPath) pkgs`.

- `bwrapFlags` - allows you to override bwrap flags (only relevant for

  downstream, non-nixpkgs projects)

- `data` - path to the dependencies lockfile (can be relative to the

  package, can be absolute). In nixpkgs, it's discouraged to have the

  lockfiles be named anything other `deps.json`, consider creating

  subdirectories if your package requires multiple `deps.json` files.

## Environment {#gradle-environment}

The Gradle setup hook accepts the following environment variables:

- `mitmCache` - the MITM proxy cache imported using `gradle.fetchDeps`

- `gradleFlags` - command-line flags to be used for every Gradle

  invocation (this simply registers a function that uses the necessary

  flags).

  - You can't use `gradleFlags` for flags that contain spaces, in that

    case you must add `gradleFlagsArray+=("-flag with spaces")` to the

    derivation's bash code instead.

  - If you want to build the package using a specific Java version, you

    can pass `"-Dorg.gradle.java.home=${jdk}"` as one of the flags.

- `gradleBuildTask` - the Gradle task (or tasks) to be used for building

  the package. Defaults to `assemble`.

- `gradleCheckTask` - the Gradle task (or tasks) to be used for checking

  the package if `doCheck` is set to `true`. Defaults to `test`.

- `gradleUpdateTask` - the Gradle task (or tasks) to be used for

  fetching all of the package's dependencies in

  `mitmCache.updateScript`. Defaults to `nixDownloadDeps`.

- `gradleUpdateScript` - the code to run for fetching all of the

  package's dependencies in `mitmCache.updateScript`. Defaults to

  running the `preBuild` and `preGradleUpdate` hooks, running the

  `gradleUpdateTask`, and finally running the `postGradleUpdate` hook.

- `gradleInitScript` - path to the `--init-script` to pass to Gradle. By

  default, a simple init script that enables reproducible archive

  creation is used.

  - Note that reproducible archives might break some builds. One example

    of an error caused by it is `Could not create task ':jar'. Replacing

    an existing task that may have already been used by other plugins is

    not supported`. If you get such an error, the easiest "fix" is

    disabling reproducible archives altogether by setting

    `gradleInitScript` to something like `writeText

    "empty-init-script.gradle" ""`

- `enableParallelBuilding` / `enableParallelChecking` /

  `enableParallelUpdating` - pass `--parallel` to Gradle in the

  build/check phase or in the update script. Defaults to true. If the

  build fails for mysterious reasons, consider setting this to false.

- `dontUseGradleConfigure` / `dontUseGradleBuild` / `dontUseGradleCheck`

  \- force disable the Gradle setup hook for certain phases.

  - Note that if you disable the configure hook, you may face issues

    such as `Failed to load native library 'libnative-platform.so'`,

    because the configure hook is responsible for initializing Gradle.

, stdenv

, fetchFromGitHub

, gradle_7

, perl

, makeWrapper

, writeText

, jdk

, gsettings-desktop-schemas

}:

let

  gradle = gradle_7;

in

stdenv.mkDerivation (finalAttrs: {

  pname = "mucommander";

  version = "1.3.0-1";

  src = fetchFromGitHub {

    owner = "mucommander";

    repo = "mucommander";

    rev = version;

    rev = finalAttrs.version;

    sha256 = "sha256-rSHHv96L2EHQuKBSAdpfi1XGP2u9o2y4g1+65FHWFMw=";

  };

    # there is no .git anyway

    substituteInPlace build.gradle \

      --replace "git = grgit.open(dir: project.rootDir)" "" \

      --replace "id 'org.ajoberstar.grgit' version '3.1.1'" "" \

      --replace "revision = git.head().id" "revision = '${version}'"

  '';

  # fake build to pre-download deps into fixed-output derivation

  deps = stdenv.mkDerivation {

    pname = "mucommander-deps";

    inherit version src postPatch;

    nativeBuildInputs = [ gradle_7 perl ];

    buildPhase = ''

      export GRADLE_USER_HOME=$(mktemp -d)

      gradle --no-daemon tgz

    '';

    # perl code mavenizes pathes (com.squareup.okio/okio/1.13.0/a9283170b7305c8d92d25aff02a6ab7e45d06cbe/okio-1.13.0.jar -> com/squareup/okio/okio/1.13.0/okio-1.13.0.jar)

    # reproducible by sorting

    installPhase = ''

      find $GRADLE_USER_HOME/caches/modules-2 -type f -regex '.*\.\(jar\|pom\)' \

        | LC_ALL=C sort \

        | perl -pe 's#(.*/([^/]+)/([^/]+)/([^/]+)/[0-9a-f]{30,40}/([^/\s]+))$# ($x = $2) =~ tr|\.|/|; "install -Dm444 $1 \$out/$x/$3/$4/$5" #e' \

        | sh

      # copy maven-metadata.xml for commons-codec

      # thankfully there is only one xml

      cp $GRADLE_USER_HOME/caches/modules-2/resources-2.1/*/*/maven-metadata.xml $out/commons-codec/commons-codec/maven-metadata.xml

      --replace "revision = git.head().id" "revision = '${finalAttrs.version}'"

  '';

    outputHashAlgo = "sha256";

    outputHashMode = "recursive";

    outputHash = "sha256-9tCcUg7hDNbkZiQEWtVRsUUfms73aU+vt5tQsfknM+E=";

  };

in

stdenv.mkDerivation rec {

  pname = "mucommander";

  inherit version src postPatch;

  nativeBuildInputs = [ gradle_7 perl makeWrapper ];

  nativeBuildInputs = [ gradle makeWrapper ];

  # Point to our local deps repo

  gradleInit = writeText "init.gradle" ''

    logger.lifecycle 'Replacing Maven repositories with ${deps}...'

    gradle.projectsLoaded {

      rootProject.allprojects {

        buildscript {

          repositories {

            clear()

            maven { url '${deps}' }

          }

        }

        repositories {

          clear()

          maven { url '${deps}' }

        }

      }

    }

    settingsEvaluated { settings ->

      settings.pluginManagement {

        repositories {

          maven { url '${deps}' }

        }

      }

    }

  '';

  mitmCache = gradle.fetchDeps {

    inherit (finalAttrs) pname;

    data = ./deps.json;

  };

  buildPhase = ''

    export GRADLE_USER_HOME=$(mktemp -d)

  __darwinAllowLocalNetworking = true;

    gradle --offline --init-script ${gradleInit} --no-daemon tgz

  '';

  gradleBuildTask = "tgz";

  installPhase = ''

    mkdir -p $out/share/mucommander

    platforms = platforms.all;

    mainProgram = "mucommander";

  };

}

})

From 33d8de9ccce7eecb12542e0fc11131b5101e1aa8 Mon Sep 17 00:00:00 2001

From: Maximilian Bosch <maximilian@mbosch.me>

Date: Sat, 26 Feb 2022 12:33:13 +0100

Subject: [PATCH] Fetch buildconfig during gradle build inside Nix FOD

---

 build.gradle | 7 ++++++-

 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/build.gradle b/build.gradle

index ea3fea1..01e444d 100644

--- a/build.gradle

+++ b/build.gradle

@@ -82,6 +82,9 @@ static String getVersion() {

 repositories {

     maven {url "https://gitlab.com/api/v4/groups/6853927/-/packages/maven"} // https://gitlab.com/groups/signald/-/packages

+    maven {

+      url "https://plugins.gradle.org/m2/"

+    }

     mavenCentral()

 }

@@ -101,6 +104,8 @@ dependencies {

     implementation 'io.prometheus:simpleclient:0.16.0'

     implementation 'io.prometheus:simpleclient_hotspot:0.16.0'

     implementation 'io.prometheus:simpleclient_httpserver:0.16.0'

+    implementation 'com.github.gmazzo.buildconfig:com.github.gmazzo.buildconfig.gradle.plugin:3.0.3'

+    implementation 'org.jetbrains.kotlin:kotlin-scripting-jvm:1.7.10'

     implementation 'com.squareup.okhttp3:logging-interceptor:4.10.0'

     implementation 'io.sentry:sentry:6.11.0'

     testImplementation 'org.junit.jupiter:junit-jupiter:5.9.0'

@@ -170,4 +175,4 @@ allprojects {

 runtime {

     options = ['--strip-java-debug-attributes', '--compress', '2', '--no-header-files', '--no-man-pages']

     modules = ['java.base', 'java.management', 'java.naming', 'java.sql', 'java.xml', 'jdk.crypto.ec', 'jdk.httpserver', 'java.desktop', 'jdk.unsupported']

-}

\ No newline at end of file

+}

-- 

2.38.3

From 4bf0aef4003f7494103a93ae1c2957b2cd32bb59 Mon Sep 17 00:00:00 2001

From: Maximilian Bosch <maximilian@mbosch.me>

Date: Sat, 26 Feb 2022 12:36:15 +0100

Subject: [PATCH 2/2] buildconfig/local deps fixes

---

 build.gradle | 26 ++++++++++++++++++--------

 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/build.gradle b/build.gradle

index ea3fea1..24415d8 100644

--- a/build.gradle

+++ b/build.gradle

@@ -10,11 +10,21 @@ import org.gradle.nativeplatform.platform.internal.ArchitectureInternal

 import org.gradle.nativeplatform.platform.internal.DefaultNativePlatform

 import org.gradle.nativeplatform.platform.internal.OperatingSystemInternal

+buildscript {

+  repositories {

+    maven {

+      url(uri("@deps@"))

+    }

+  }

+  dependencies {

+    classpath "com.github.gmazzo:gradle-buildconfig-plugin:3.0.3"

+  }

+}

+

 plugins {

-   id 'com.github.gmazzo.buildconfig' version '3.0.3'

-   id 'org.beryx.runtime' version '1.12.7'

    id 'application'

 }

+apply plugin: "com.github.gmazzo.buildconfig"

 compileJava.options.encoding = 'UTF-8'

@@ -82,7 +92,10 @@ static String getVersion() {

 repositories {

     maven {url "https://gitlab.com/api/v4/groups/6853927/-/packages/maven"} // https://gitlab.com/groups/signald/-/packages

-    mavenCentral()

+    mavenLocal()

+    maven {

+      url uri("@deps@")

+    }

 }

 dependencies {

@@ -101,6 +114,8 @@ dependencies {

     implementation 'io.prometheus:simpleclient:0.16.0'

     implementation 'io.prometheus:simpleclient_hotspot:0.16.0'

     implementation 'io.prometheus:simpleclient_httpserver:0.16.0'

+    implementation 'com.github.gmazzo.buildconfig:com.github.gmazzo.buildconfig.gradle.plugin:3.0.3'

+    implementation 'org.jetbrains.kotlin:kotlin-scripting-jvm:1.7.10'

     implementation 'com.squareup.okhttp3:logging-interceptor:4.10.0'

     implementation 'io.sentry:sentry:6.11.0'

     testImplementation 'org.junit.jupiter:junit-jupiter:5.9.0'

@@ -166,8 +181,3 @@ allprojects {

         }

     }

 }

-

-runtime {

-    options = ['--strip-java-debug-attributes', '--compress', '2', '--no-header-files', '--no-man-pages']

-    modules = ['java.base', 'java.management', 'java.naming', 'java.sql', 'java.xml', 'jdk.crypto.ec', 'jdk.httpserver', 'java.desktop', 'jdk.unsupported']

-}

\ No newline at end of file

-- 

2.38.3