RPM plugin generates RPM with bad SHA256 digests, won't install on RHEL8 with FIPS enabled

[jelion]

A simple Gradle project that generates a relatively simple RPM sometimes builds RPMs with 'bad' checksums. Such RPMs will not install on a RHEL 8 system with FIPS mode enabled. I have seen this error with 9.1.1, the most recent version I have seen public documentation for. I did notice a version 11.0.0 available in some places; the error occurs with that version too.

Sometimes the RPM will install (on RHEL 8.6 with FIPS compliance checking enabled), other times the RPM will fail to install with an error similar to the following:

[cloud-user@paas-installer ~]$ sudo rpm --install ~/testpackage-1.0.0-1.noarch.rpm
        package testpackage-0:1.0.0-1.noarch does not verify: Payload SHA256 digest: BAD (Expected 747e511743005c7c0d65a279cca436f156e63585e8023cbb41d5326ae4f47604 != f6accca179a01e379434ffb1190fdca887d535d65615a45f19ebeb19778eab3a)

The RPM can be examined with the --checksig switch. A "good" RPM looks like:

[cloud-user@paas-installer foo]$ rpm --checksig -v ~/testpackage-1.0.0-1.noarch.rpm
/home/cloud-user/testpackage-1.0.0-1.noarch.rpm:
    Header SHA256 digest: OK
    Header SHA1 digest: OK
    Payload SHA256 digest: OK

A bad one looks like:

[cloud-user@paas-installer foo]$ rpm --checksig -v ~/testpackage-1.0.0-1.noarch.rpm
/home/cloud-user/testpackage-1.0.0-1.noarch.rpm:
    Header SHA256 digest: OK
    Header SHA1 digest: OK
    Payload SHA256 digest: BAD (Expected 747e511743005c7c0d65a279cca436f156e63585e8023cbb41d5326ae4f47604 != f6accca179a01e379434ffb1190fdca887d535d65615a45f19ebeb19778eab3a)
    MD5 digest: NOTFOUND

Running 'gradle build' on the attached build script, with a single trivial script in a scripts subfolder, produces an RPM that is sometimes good and sometimes bad. They tend to come in cycles; I may get 20 good RPMs in a row, and then 3 bad ones in a row. I have tried building with '--no-daemon'; deleting the output RPM, and even the entire build folder between builds - but have not been able to reliably produce a 'bad' RPM (or a 'good' one). It just seems to be that some are bad, and some are good. (Most are good; it may be telling that I have yet to notice bad RPMs in a completely-clean project initial build.

We are building with Gradle 7.6, on Windows 10 (AMD 64) and Java 8. We used the plugin for 1 1/2 years on RHEL 7 with no issues (FIPS was not enabled on those environments; in performing the --checksig test on a number of releases we found that 29 of 290 RPMs had bad signatures.) Our Linux environment is:

[root@paas-installer ~] uname -a
Linux paas-installer.jkepaas.gov 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux
[root@paas-installer cloud-user]# fips-mode-setup --check
FIPS mode is enabled.

(The RPM --checksig test should work whether FIPS is enabled or not.)

I have not been able to paste to this ticket, so my build file is pasted below. The test project also has a single file in scripts with trivial content.
build.gradle:

buildscript {
  repositories { maven { url 'https://plugins.gradle.org/m2' }}
  dependencies {
    classpath 'com.netflix.nebula:gradle-ospackage-plugin:9.1.1'
  }
}

apply plugin: 'nebula.ospackage'
import org.redline_rpm.header.RpmType
import org.redline_rpm.header.Architecture
import org.redline_rpm.header.Os

task rpm(type: Rpm) {
  packageName     'testpackage'
  description     'Test Package'
  version         '1.0.0'
  release         '1'

  type            RpmType.BINARY
  arch            Architecture.NOARCH
  os              Os.LINUX

  user            'root'
  permissionGroup 'root'
  dirMode         0755
  fileMode        0755

  directory('/opt/test', 0755)

  from ('scripts') {
    into '/opt/testpackage/bin'
  }
}

build.dependsOn rpm

[Hendrik-H]

I'm seeing the issue as well but so far only on s390x and not x86_64. The payload signing support was added in redline 1.2.10, which was picked up here: a942e96 So the problem should exist since 8.6.1.

[patricklucas]

I've just started to use this plugin and very quickly ran into this on an almost trivial rpm—just a single preinstall script.

[DanielThomas]

@patricklucas do you also experience this as an intermittent failure, or is it reproducible?

produces an RPM that is sometimes good and sometimes bad

We don't use rpms ourselves, so we'll find it difficult to find the time to look into this. But if someone has time to troubleshoot and fix, we'd be happy to merge/release from a PR.

[mricciuti]

Hi @DanielThomas , pinging you as there are others PRs waiting to be checked for quite some time, maybe you would be happy to check/merge those as well? ( eg.: #414 , #417 , #424 , and some others)

[patricklucas]

@DanielThomas I've only encountered it once so far, but at the same time I've only used the plugin for a few days, so statistically-speaking...

If it happens again I'll report back here.

[TwilightCitizen]

Just adding a "Me, too!" to this thread. The issue has occurred intermittently, but frequently enough to transition from annoying to mildly blocking. First experienced it on a 9.X release, though 11.X releases continue to exhibit the issue, too.

[DanielThomas]

The only explanation I can offer is you have a task or external process modifying a file after the digest is computed, but before the payload is processed.

We just configure the Builder, the digest and signatures are completely handled by redline:

• https://github.com/nebula-plugins/gradle-ospackage-plugin/blob/7f0a8aa0d31176788590b9cc86bc3c6a655741aa/src/main/groovy/com/netflix/gradle/plugins/rpm/RpmCopyAction.groovy
• https://github.com/craigwblake/redline/blob/15afff553f65e3c9d0e9d904fd1e8ce46c4d1515/src/main/java/org/redline_rpm/Builder.java#L1349C62-L1356

[TwilightCitizen]

We were able to resolve our issue by modifying our Gradle build script to use the Redline RPM package directly. Initially, I ran into quite a learning curve with it. Mainly, I was on the struggle bus getting the logic right to recursively add files from source directories to destination directories while getting the paths correct. While I was sorting this out, we were still indeed getting bad hashes. Now, that this is working correctly, we are consistently getting good hashes in our RPM.

I am not exactly sure what the underlying issue is. Is the Nebula OS Package plugin is doing something "wrong" like I was with the Redline RPM package, or we are just getting lucky at the moment with many RPMs built with good hashes in a row? That was the case with the Nebula OS Package plugin for a while. Time will tell.

One thing that pressed us to take this course of action is that the intermittent bad hashes we were getting with the Nebula OS Package plugin became far more commonplace, where we were then getting only intermittent successes.

I will add that the issue only cropped up when building via a Jenkins pipeline. Perhaps Jenkins is just doing something with paths that is unexpected in the Nebula OS Package plugin's logic.

I unfortunately don't have a lot of time to look more deeply into this at the moment.

[micheljung]

Keeps happening to us when an RPM file already exists and is rebuilt, never happens when we delete the existing RPM before rebuilding it.

[pevdh]

Our project also ran into this issue recently. Based off the comment left by @micheljung I suspect there's an issue in com.netflix.gradle.plugins.rpm.RpmCopyAction.end():

    @Override
    protected void end() {
        RandomAccessFile rpmFile
        try {
            rpmFile = new RandomAccessFile(task.getArchiveFile().get().asFile, "rw")
            builder.build(rpmFile.getChannel())
            logger.info 'Created rpm {}', rpmFile
        } finally {
            if (rpmFile != null) {
                rpmFile.close()
            }
        }
    }

I might be wrong, but I think the plugin needs to either truncate or delete the existing file before calling builder.build(), otherwise the builder might create a corrupted RPM because it will only partially overwrite the existing file.

As @micheljung implies a workaround is:

    doFirst {
        File rpmFile = archiveFile.get().getAsFile()
        if (rpmFile.exists()) {
            rpmFile.delete()
        }
    }

[jelion]

Thank you Michael Jung. Your explanation seems reasonable in that it would explain why the problem seems to occur only on rebuilds. I have not been able to clone, build, and test the fix from your MR but I hope I will be able to do that soon (which alas could be after the new year, but I've seen this and I'm grateful for the contribution.)

[micheljung]

@jelion being after the new year, would you mind merging and releasing #449 to fix this? 😃 (or maybe rather @rpalcolea )