About

Gradle plugin and helpers to assist with writing tests for other gradle plugins.

Using the Plugin

apply plugin: 'com.palantir.gradle-plugin-testing'

The plugin automatically adds a testImplementation dependency on the com.palantir.gradle.plugintesting:plugin-testing-core library so that the helper classes are available in tests.

Primary Functionality

Automatic suppression of gradle deprecation warnings

The nebula-test framework will automatically fail a test if it detects gradle deprecation messages in the output. Unfortunately, this will fail if another plugin used in a test is using deprecated gradle components even if the actual plugin under test is clean. The plugin-testing plugin will automatically set the ignoreDeprecations system property so that the nebula-test framework will ignore deprecation messages. Use other mechanisms, such as java deprecation linting, to detect and fix deprecations in the plugin under test.

This behavior can be overridden using the gradleTestUtils extension. e.g.

gradleTestUtils {
    ignoreGradleDeprecations = false
}

Resolution of dependencies in generated build files

Integration tests for Gradle plugins often write build files which include other plugins. The version of those included plugins is hardcoded in some way, either directly in the test, e.g.

//...within a nebula spec...
  buildFile << """
     buildscript {
         repositories {
             mavenCentral()
         }
         dependencies {
             classpath 'com.palantir.gradle.conjure:gradle-conjure:0.0.1'
         }
     }
     dependencies {
         implementation 'com.palantir.conjure.java:conjure-lib:0.0.1'
     }
     """

or in some "Versions" class that has a listing of all the dependencies the test uses. e.g.

final class TestPluginVersions {
    static final String CONJURE_JAVA = "com.palantir.conjure.java:conjure-java:5.7.1";
    static final String CONJURE = "com.palantir.conjure:conjure:4.10.1";
}

Once these tests are written, the versions of the plugins are often not updated, even when the project under test keeps its dependencies of those plugins up-to-date. This can cause tests to fail when the plugin is updated not because the plugin is bad, but because there is an incompatibility in the old versions of plugins used in the integration tests. This can often happen with Gradle version bumps.

When applied to a project, the gradle-plugin-testing plugin scans the testRuntimeClasspath configuration for the project and passes all dependencies to the test task as a system property. The version of the dependencies can then be resolved when tests are run and written into generated files. e.g.

import static com.palantir.gradle.plugintesting.TestDependencyVersions.resolve
import nebula.test.IntegrationSpec

class HelloWorldSpec extends IntegrationSpec {
   def setup() {
       //language=gradle
       buildFile << """
        buildscript {
            repositories {
                mavenCentral()
            }
            dependencies {
                classpath '${resolve('com.palantir.gradle.conjure:gradle-conjure')}'
            }
        }
        dependencies {
            implementation '${resolve('com.palantir.conjure.java:conjure-lib')}'
        }
        """
   }

Adding Test Only Plugin Dependencies

You may find yourself needing to resolve a dependency on a plugin that you do not directly consume as part of your implementation. In these scenarios you will need to add additional, test only, dependencies. For example, leveraging Gradle consistent versions:

versions.props:

# Nebula test dependencies (comment is nice, but certainly not necessary)
com.palantir.sls-packaging:gradle-sls-packaging = 7.69.0

build.gradle:

dependencies {
    // Other dependencies...
    
    // Note: 'testRuntimeOnly' is used here intentionally as the Baseline 
    //   'checkUnusedDependencies' task is (currently) unaware of the 'resolve'
    //   method this plugin provides. Given the scenario described above, we do not use this plugin
    //   directly and thus 'testImplementation' triggers a warning to remove this dependency
    testRuntimeOnly 'com.palantir.sls-packaging:gradle-sls-packaging'
}

Resolution of Gradle versions to test against

Similarly, tests may hardcode versions of Gradle that they need to stay compatible with. These versions also get stale and PRs start failing for the inverse reason of the above - the code in the plugin or a dependency of it is updated and is no longer compatible with an old version of Gradle. For example, attempting to update jackson libraries from 2.15.0 -> 2.17.0 would fail if a test tried to run on Gradle versiosn < 7.6.4 (when compatibility with jackson 2.17.0 was fixed).

Setting Gradle test versions

There are two ways to set which Gradle versions to test against:

• setting the versions in the gradle/gradle-test-versions.yml file
• setting gradleVersions in the the gradleTestUtils extension (this is deprecated) When using @GradlePluginTests, the versions of Gradle used will be the union of the versions above.

If you haven't set either, tests will fall back to using the Gradle version of the test driver.

The gradle/gradle-test-versions.yml file

An example file:

major-versions:
  8: 8.14.3
  9: 9.2.0-rc1
extra-versions:
- 8.8
- 8.14.2

The motivation for having a separate config file is to more easily allow an automated process of bumping the versions of Gradle we test against, whilst also allowing users to pick out specific versions to test against. Concretely, an automated process would bump versions in the major-versions blob, whilst leaving extra-versions alone. When support for a Gradle major version is phased out, it can be just removed from this file in one fell swoop.

Note

This file is global, and affects every project that is using this plugin.

[Deprecated] Setting gradleVersions in the gradleTestUtils extension

You can set Gradle versions to test by setting gradleVersions property on the gradleTestUtils extension. For example, to test against Gradle versions 7.6.4 and 8.8, you would do:

gradleTestUtils {
  gradleVersions = ['7.6.4', '8.8']
}

Warning

This is deprecated because it's merges together the intent of a user wanting a specific version and also the intent of testing against the latest versions of Gradle. How does an automated process know if the user wanted to test this specific version which also happens to be the latest version?## Accessing Gradle test versions

Java tests with @GradlePluginTests

Tests annotated with @GradlePluginTests automatically run with all the versions specified in the config file.

[Deprecated] Groovy tests with spock e.g. IntegrationSpec

Access Gradle test versions via the GradleTestVersions#gradleVersionForTests method

import nebula.test.IntegrationSpec
import com.palantir.gradle.plugintesting.GradleTestVersions

class HelloWorldSpec extends IntegrationSpec {
    def 'runs on version of gradle: #version'() {
        when:
        gradleVersion = version

        then:
        def result = runTasks('someTask')
        result.success

        where:
        version << GradleTestVersions.gradleVersionsForTests
    }
}

Gradle Plugin Testing Framework

The gradle-plugin-testing framework provides a modern, Java-based testing approach for Gradle plugins. Built on Gradle TestKit, it features JUnit 5 integration, fluent APIs, and automatic multi-version testing across different Gradle releases.

For detailed information on how to write tests, see the Testing Guide.