Merge pull request #169 from jonesbusy/feature/init-integration-tests

Init testcontainer integration tests using Ansible agent

Author: jonesbusy

Diff for: Jenkinsfile

@@ -4,7 +4,7 @@
 */
 buildPlugin(
   forkCount: '1C', // run this number of tests in parallel for faster feedback.  If the number terminates with a 'C', the value will be multiplied by the number of available CPU cores
-  useContainerAgent: true, // Set to `false` if you need to use Docker for containerized tests
+  useContainerAgent: false, // Set to `false` if you need to use Docker for containerized tests
   configurations: [
     [platform: 'linux', jdk: 21],
     [platform: 'windows', jdk: 17],

Diff for: README.md

@@ -31,6 +31,8 @@ Jenkins jobs.
 
 3.  Repeat for any additional desired installations
 
+There is no automatic ansible installation possible using Global Tools.
+
 ### OS User PATH
 
 Ansible can also be added to the PATH user used by the Jenkins executor
@@ -40,6 +42,18 @@ guide.
 
 ------------------------------------------------------------------------
 
+## Supported versions
+
+The plugin is tested against supported ansible-core versions (https://endoflife.date/ansible-core). It might work with older versions, but this is not guaranteed.
+
+See `PipelineTest.java`
+
+```java
+private static Stream<String> ansibleVersions() {
+    return Stream.of("2.14.13", "2.15.8", "2.16.2");
+}
+```
+
 ## Adhoc
 
 [Adhoc commands](http://docs.ansible.com/ansible/latest/intro_adhoc.html) allow
@@ -289,122 +303,9 @@ a "Secret text" or a "Secret file".
 
 ------------------------------------------------------------------------
 
-## Open Issues
-
-[View issues in Jira](https://issues.jenkins.io/secure/IssueNavigator.jspa?reset=true&jqlQuery=component%20=%20ansible-plugin%20AND%20status%20in%20%28Open,%20%22In%20Progress%22,%20Reopened%29&tempMax=1000&src=confmacro)
-
-------------------------------------------------------------------------
-
 ## Changelog
 
-#### Version 1.0 (26 March 2018)
-
-*  [Fix security issue](https://jenkins.io/security/advisory/2018-03-26/#SECURITY-630):
-            Do not disable host key verification by default. **This may
-            break existing configurations as host key verification will
-            be enabled everywhere by default.**
-
-#### Version 0.8.0 (16 Jan 2018)
-
-*  Add support for Ansible Vault
-            [JENKINS-48499](https://issues.jenkins.io/browse/JENKINS-48499)
-*   Add hostKeyChecking option to pipeline [JENKINS-42445](https://issues.jenkins.io/browse/JENKINS-42445)
-
-#### Version 0.6.2 (3 Jan 2017)
-
-*  Fix blocker bug when launched from a pipeline
-            [JENKINS-40780](https://issues.jenkins.io/browse/JENKINS-40780)
-
-#### Version 0.6.1 (1 Jan 2017)
-
-*  Use latest parent project definition in order to deploy
-            plugin (thanks
-            to [alecharp](https://github.com/alecharp) for the help and
-            the PR)
-
-#### Version 0.6 (31 Dec 2016)
-
-**WARN: 0.6.x version will be the last one to support Jenkins 1.xxx and
-Ansible 1.x - The 0.7.x and next releases will require Jenkins 2.32.1
-(or higher) and Ansible 2.2 (or higher)**
-
-*  Add a "do not specify" option for
-            inventory [JENKINS-34627](https://issues.jenkins.io/browse/JENKINS-34627)
-* Support inventoryContent in pipeline (thanks
-            to [leewin12](https://github.com/leewin12) for the PR)
-* Add support of extra variables in jobdsl (thanks
-            to [pawbur](https://github.com/pawbur) for the PR)
-* Support empty forks (number of parallel processes)
-            parameter [JENKINS-39438](https://issues.jenkins.io/browse/JENKINS-39438)
-* Escape '%' character in private key path (thanks
-            to [ewollesen](https://github.com/ewollesen) for the PR)
-* Omit ansible option when expanded environment variable is
-            empty (thanks to [vjestin](https://github.com/vjestin) for
-            the PR)
-* Add the --forks parameter configurable in pipeline step
-            (thanks to
-            [anguswilliams](https://github.com/anguswilliams) for the
-            PR)
-* Fix usage of environment variable in ansiblePlaybook
-            pipeline step (thanks to
-            [thomasKalmar](https://github.com/thomasKalmar)
-            and [barthorre](https://github.com/barthorre) for the
-            PR) [JENKINS-38289](https://issues.jenkins.io/browse/JENKINS-38289)
-
-#### Version 0.5 (5 May 2016)
-
-*  Add support for ansible extra variables
-            [JENKINS-29863](https://issues.jenkins.io/browse/JENKINS-29863)
-* Improve Pipeline plugin
-            integration [JENKINS-32911](https://issues.jenkins.io/browse/JENKINS-32911)
-* Add the possibility to use the default inventory file
-            (thanks to Johann Schmitz for the PR)
-* Add colorized output in pipeline jobs (thanks to
-            Kirill Merkushev for the PR)
-* Make Jenkins build variables available as environment
-            variables for ansible (thanks to Kevin Mooney for the
-            PR) [JENKINS-29284](https://issues.jenkins.io/browse/JENKINS-29284)
-
-#### Version 0.4 (25 December 2015)
-
-*  Support for password protected SSH
-            keys [JENKINS-30656](https://issues.jenkins.io/browse/JENKINS-30656)
-* Initial support for the workflow
-            plugin [JENKINS-30398](https://issues.jenkins.io/browse/JENKINS-30398)
-* Add support for Job DSL plugin (thanks to Kirill Merkushev
-            for the
-            PR) [JENKINS-31790](https://issues.jenkins.io/browse/JENKINS-31790)
-
-#### Version 0.3.1 (15 July 2015)
-
-*  Fix execution on slave
-            nodes [JENKINS-29294](https://issues.jenkins.io/browse/JENKINS-29294)
-
-#### Version 0.3 (20 June 2015)
-
-*  Add support for password based SSH authentication (with
-            sshpass)
-* Environment variables can be used in Module and Module
-            arguments text field in Ad-hoc command builder
-* Environment variables can be used in inline inventory text
-            box
-            [JENKINS-28547](https://issues.jenkins.io/browse/JENKINS-28547)
-
-#### Version 0.2 (11 May 2015)
-
-*  Fix NullPointerException when no credentials are selected
-* Fix --skippedTags parameter configuration which was ignored
-* Fix NullPointerException and print an error message in the
-            build console when the inventory is not set in the job
-            configuration
-
-#### Version 0.1 (01 May 2015)
-
-*  Initial version
-
-This plugin gives the possibility to run [Ansible](http://www.ansible.com/) ad-hoc command or playbooks as a build step.
-
-[![Build Status](https://ci.jenkins.io/buildStatus/icon?job=Plugins/ansible-plugin/master)](https://ci.jenkins.io/job/Plugins/job/ansible-plugin/job/master/)
+Changelog is now published on GitHub release.
 
 ## Using Jenkins Build and Environment Variables
 

Diff for: pom.xml

@@ -30,6 +30,10 @@
     <gitHubRepo>jenkinsci/ansible-plugin</gitHubRepo>
     <jenkins.version>2.426.2</jenkins.version>
     <spotless.check.skip>false</spotless.check.skip>
+
+    <!-- Test dependencies version -->
+    <testcontainer.version>1.19.3</testcontainer.version>
+
   </properties>
   <dependencyManagement>
     <dependencies>
@@ -66,12 +70,12 @@
       <groupId>org.jenkins-ci.plugins</groupId>
       <artifactId>ssh-credentials</artifactId>
     </dependency>
+    <!-- Test dependencies -->
     <dependency>
       <groupId>org.jenkins-ci.plugins.workflow</groupId>
       <artifactId>workflow-step-api</artifactId>
       <optional>true</optional>
     </dependency>
-    <!-- Test -->
     <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
@@ -83,16 +87,15 @@
       <scope>test</scope>
     </dependency>
     <dependency>
-      <groupId>org.jenkins-ci.plugins.workflow</groupId>
-      <artifactId>workflow-basic-steps</artifactId>
+      <groupId>org.jenkins-ci.plugins</groupId>
+      <artifactId>ssh-slaves</artifactId>
       <scope>test</scope>
     </dependency>
     <dependency>
       <groupId>org.jenkins-ci.plugins.workflow</groupId>
       <artifactId>workflow-cps</artifactId>
       <scope>test</scope>
     </dependency>
-    <!-- Test plugins -->
     <dependency>
       <groupId>org.jenkins-ci.plugins.workflow</groupId>
       <artifactId>workflow-job</artifactId>
@@ -108,6 +111,32 @@
       <artifactId>mockito-core</artifactId>
       <scope>test</scope>
     </dependency>
+    <dependency>
+      <groupId>org.testcontainers</groupId>
+      <artifactId>junit-jupiter</artifactId>
+      <version>${testcontainer.version}</version>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.testcontainers</groupId>
+      <artifactId>testcontainers</artifactId>
+      <version>${testcontainer.version}</version>
+      <scope>test</scope>
+      <exclusions>
+        <exclusion>
+          <groupId>org.apache.commons</groupId>
+          <artifactId>commons-compress</artifactId>
+        </exclusion>
+        <exclusion>
+          <groupId>org.jetbrains</groupId>
+          <artifactId>annotations</artifactId>
+        </exclusion>
+        <exclusion>
+          <groupId>org.slf4j</groupId>
+          <artifactId>slf4j-api</artifactId>
+        </exclusion>
+      </exclusions>
+    </dependency>
   </dependencies>
   <repositories>
     <repository>

Diff for: src/main/java/org/jenkinsci/plugins/ansible/AnsibleAdHocCommandBuilder.java

@@ -239,7 +239,9 @@ public void perform(
                             ? CredentialsProvider.findCredentialById(vaultCredentialsId, StandardCredentials.class, run)
                             : null);
             invocation.setVaultTmpPath(
-                    StringUtils.isNotBlank(vaultTmpPath) ? new FilePath(new File(vaultTmpPath)) : null);
+                    StringUtils.isNotBlank(vaultTmpPath)
+                            ? new FilePath(computer.getChannel(), new File(vaultTmpPath).getAbsolutePath())
+                            : null);
             invocation.setExtraVars(extraVars);
             invocation.setAdditionalParameters(additionalParameters);
             invocation.setDisableHostKeyCheck(disableHostKeyChecking);

Diff for: src/main/java/org/jenkinsci/plugins/ansible/AnsiblePlaybookBuilder.java

@@ -266,6 +266,7 @@ public void perform(
             throws InterruptedException, IOException {
         try {
             CLIRunner runner = new CLIRunner(run, ws, launcher, listener);
+            Computer computer = node.toComputer();
             String exe = AnsibleInstallation.getExecutable(
                     ansibleName, AnsibleCommand.ANSIBLE_PLAYBOOK, node, listener, envVars);
             AnsiblePlaybookInvocation invocation = new AnsiblePlaybookInvocation(exe, run, ws, listener, envVars);
@@ -295,7 +296,9 @@ public void perform(
                                     run)
                             : null);
             invocation.setVaultTmpPath(
-                    StringUtils.isNotBlank(vaultTmpPath) ? new FilePath(new File(vaultTmpPath)) : null);
+                    StringUtils.isNotBlank(vaultTmpPath)
+                            ? new FilePath(computer.getChannel(), new File(vaultTmpPath).getAbsolutePath())
+                            : null);
             invocation.setExtraVars(extraVars);
             invocation.setAdditionalParameters(additionalParameters);
             invocation.setDisableHostKeyCheck(disableHostKeyChecking);

Diff for: src/main/java/org/jenkinsci/plugins/ansible/AnsibleVaultBuilder.java

@@ -125,6 +125,7 @@ public void perform(
             throws InterruptedException, IOException {
         try {
             CLIRunner runner = new CLIRunner(run, ws, launcher, listener);
+            Computer computer = node.toComputer();
             String exe = AnsibleInstallation.getExecutable(
                     ansibleName, AnsibleCommand.ANSIBLE_VAULT, node, listener, envVars);
             AnsibleVaultInvocation invocation = new AnsibleVaultInvocation(exe, run, ws, listener, envVars);
@@ -144,7 +145,9 @@ public void perform(
                                     run)
                             : null);
             invocation.setVaultTmpPath(
-                    StringUtils.isNotBlank(vaultTmpPath) ? new FilePath(new File(vaultTmpPath)) : null);
+                    StringUtils.isNotBlank(vaultTmpPath)
+                            ? new FilePath(computer.getChannel(), new File(vaultTmpPath).getAbsolutePath())
+                            : null);
             invocation.setContent(content);
             invocation.setInput(input);
             invocation.setOutput(output);

Diff for: src/main/java/org/jenkinsci/plugins/ansible/Utils.java

@@ -50,6 +50,7 @@ static FilePath createSshKeyFile(FilePath key, FilePath tmpPath, SSHUserPrivateK
     static FilePath createSshAskPassFile(
             FilePath script, FilePath tmpPath, SSHUserPrivateKey credentials, boolean inThisDir)
             throws IOException, InterruptedException {
+        tmpPath.mkdirs();
         StringBuilder sb = new StringBuilder();
         sb.append("#! /bin/sh\n").append("/bin/echo \"" + Secret.toString(credentials.getPassphrase()) + "\"");
         script = tmpPath.createTextTempFile("ssh", ".sh", sb.toString(), inThisDir);
@@ -69,6 +70,7 @@ static FilePath createSshAskPassFile(
     static FilePath createVaultPasswordFile(FilePath key, FilePath tmpPath, FileCredentials credentials)
             throws IOException, InterruptedException {
         try (InputStream content = credentials.getContent()) {
+            tmpPath.mkdirs();
             key = tmpPath.createTempFile("vault", ".password");
             key.copyFrom(content);
             key.chmod(0400);
@@ -87,6 +89,7 @@ static FilePath createVaultPasswordFile(FilePath key, FilePath tmpPath, FileCred
      */
     static FilePath createVaultPasswordFile(FilePath key, FilePath tmpPath, StringCredentials credentials)
             throws IOException, InterruptedException {
+        tmpPath.mkdirs();
         key = tmpPath.createTextTempFile(
                 "vault", ".password", credentials.getSecret().getPlainText(), true);
         key.chmod(0400);

Diff for: src/test/java/org/jenkinsci/plugins/ansible/AnsibleAdHocCommandInvocationTest.java

@@ -6,16 +6,12 @@
 import static org.hamcrest.Matchers.hasEntry;
 import static org.mockito.Mockito.*;
 
-import com.cloudbees.jenkins.plugins.sshcredentials.SSHUserPrivateKey;
-import com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials;
 import hudson.EnvVars;
 import hudson.model.AbstractBuild;
 import hudson.model.BuildListener;
 import hudson.model.TaskListener;
 import hudson.util.ArgumentListBuilder;
-import hudson.util.Secret;
 import java.util.Map;
-import org.junit.Ignore;
 import org.junit.Test;
 import org.mockito.ArgumentCaptor;
 
@@ -128,66 +124,6 @@ public void secure_by_default_SEC_630() throws Exception {
         assertThat((Map<String, String>) argument.getValue(), hasEntry("ANSIBLE_FORCE_COLOR", "true"));
     }
 
-    @Test
-    @Ignore("build.getWorkspace() cannot be mocked")
-    public void should_handle_private_key_credentials() throws Exception {
-        // Given
-        Inventory inventory = new InventoryPath("/tmp/hosts");
-        SSHUserPrivateKey pkey = mock(SSHUserPrivateKey.class);
-        when(pkey.getUsername()).thenReturn("mylogin");
-        BuildListener listener = mock(BuildListener.class);
-        CLIRunner runner = mock(CLIRunner.class);
-        AbstractBuild<?, ?> build = mock(AbstractBuild.class);
-        when(build.getEnvironment(any(TaskListener.class))).thenReturn(new EnvVars());
-        AnsibleAdHocCommandInvocation invocation =
-                new AnsibleAdHocCommandInvocation("/usr/local/bin/ansible", build, listener);
-        invocation.setHostPattern("localhost");
-        invocation.setInventory(inventory);
-        invocation.setModule("ping");
-        invocation.setCredentials(pkey);
-        invocation.setForks(5);
-        // When
-        invocation.execute(runner);
-        // Then
-        ArgumentCaptor<ArgumentListBuilder> argument = ArgumentCaptor.forClass(ArgumentListBuilder.class);
-        verify(runner).execute(argument.capture(), anyMap());
-
-        assertThat(
-                argument.getValue().toString(),
-                is("/usr/local/bin/ansible localhost -i /tmp/hosts -m ping -f 5 --private-key .+ -u mylogin"));
-    }
-
-    @Test
-    @Ignore("Secret can neither be instanced nor mocked")
-    public void should_handle_password_credentials() throws Exception {
-        // Given
-        Inventory inventory = new InventoryPath("/tmp/hosts");
-        StandardUsernamePasswordCredentials password = mock(StandardUsernamePasswordCredentials.class);
-        when(password.getUsername()).thenReturn("mylogin");
-        when(password.getPassword()).thenReturn(Secret.fromString("aStrongSecretPassword"));
-        BuildListener listener = mock(BuildListener.class);
-        CLIRunner runner = mock(CLIRunner.class);
-        AbstractBuild<?, ?> build = mock(AbstractBuild.class);
-        when(build.getEnvironment(any(TaskListener.class))).thenReturn(new EnvVars());
-        AnsibleAdHocCommandInvocation invocation =
-                new AnsibleAdHocCommandInvocation("/usr/local/bin/ansible", build, listener);
-        invocation.setHostPattern("localhost");
-        invocation.setInventory(inventory);
-        invocation.setModule("ping");
-        invocation.setCredentials(password);
-        invocation.setForks(5);
-        // When
-        invocation.execute(runner);
-        // Then
-        ArgumentCaptor<ArgumentListBuilder> argument = ArgumentCaptor.forClass(ArgumentListBuilder.class);
-        verify(runner).execute(argument.capture(), anyMap());
-
-        assertThat(
-                argument.getValue().toString(),
-                is("sshpass ****** /usr/local/bin/ansible localhost -i /tmp/hosts -m ping -f 5 " + "-u"
-                        + " mylogin -k"));
-    }
-
     @Test
     public void should_handle_variables() throws Exception {
         // Given