Skip to content

Conversation

seankingyang
Copy link
Contributor

@seankingyang seankingyang commented Jul 4, 2025

WARNING: This modifies com.canonical.certification::sru-server

Description

Current secure boot testing only validates enabled states, missing checkpoints when systems have secure boot disabled. This can allow systems to pass certification with configurations that don't match manifest expectations.

1. Dual Job Coverage

  • secure-boot-enabled: Validates when manifest.has_secure_boot == 'True'
  • secure-boot-disabled: Validates when manifest.has_secure_boot != 'True'

2. Enhanced Detection

  • UEFI Systems: Detects /sys/firmware/efi/ path, uses mokutil
  • FIT Image Systems: Falls back to dumpimage for image signature analysis
  • Automatic Selection: Script chooses appropriate method automatically

Files Modified

  • providers/base/units/miscellanea/jobs.pxu - Job definitions
  • providers/base/bin/check_secure_boot_state.py - Script improvements
  • Test plans - Added job references

Usage

# Manual script usage
check_secure_boot_state.py --enable --verbose
check_secure_boot_state.py --disabled --verbose

# checkbox
checkbox-cli run com.canonical.certification::miscellanea/secure-boot-enabled
checkbox-cli run com.canonical.certification::miscellanea/secure-boot-disabled

Resolved issues

Documentation

https://warthogs.atlassian.net/browse/OEMQA-6211

Tests

Testing on My Laptop: Ubuntu 22.04 Desktop Image with Secure Boot Enabled

  • Add the manifest on it, and copy the base’s unitand bin folder provider to /usr/lib|share/checkbox-provider/.
  • Add the manifest to test it. (when the has_secure_boot set to True and false)
{
  "com.canonical.certification::has_secure_boot": True
}
  • Run the command: checkbox-cli run com.canonical.certification::miscellanea/secure-boot-enabled com.canonical.certification::miscellanea/secure-boot-disabled
  • The result shows the manifest is set to True and matches my laptop's secure boot settings:
==============[ Running job 2 / 3. Estimated time left: 0:00:01 ]===============
-------[ Verify secure boot is enabled when manifest indicates support ]--------
ID: com.canonical.certification::miscellanea/secure-boot-enabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Classic via hostnamectl
Initialized UEFISecureBootChecker with hostfs_prefix='', ubuntu_variant='UbuntuVariant.CLASSIC'
Running mokutil --sb-state
mokutil reports secure boot enabled
=== Secure Boot State Check ===
Checker: UEFI (mokutil)
Current state: enabled
PASS: Secure boot is enabled
------------------------------------------------------------------------- >8 ---
Outcome: job passed
==============[ Running job 3 / 3. Estimated time left: 0:00:00 ]===============
-----[ Verify secure boot is disabled when manifest indicates no support ]------
ID: com.canonical.certification::miscellanea/secure-boot-disabled
Category: com.canonical.plainbox::miscellanea
Job cannot be started because:
 - resource expression "manifest.has_secure_boot == 'False'" evaluates to false
Outcome: job cannot be started
  • If I set an incorrect manifest that differs from my Secure Boot settings, or do not set it, the result will be:
{
  "com.canonical.certification::has_secure_boot": False
}
==============[ Running job 2 / 3. Estimated time left: 0:00:01 ]===============
-------[ Verify secure boot is enabled when manifest indicates support ]--------
ID: com.canonical.certification::miscellanea/secure-boot-enabled
Category: com.canonical.plainbox::miscellanea
Job cannot be started because:
 - resource expression "manifest.has_secure_boot == 'True'" evaluates to false
Outcome: job cannot be started
==============[ Running job 3 / 3. Estimated time left: 0:00:00 ]===============
-----[ Verify secure boot is disabled when manifest indicates no support ]------
ID: com.canonical.certification::miscellanea/secure-boot-disabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Classic via hostnamectl
Initialized UEFISecureBootChecker with hostfs_prefix='', ubuntu_variant='UbuntuVariant.CLASSIC'
Running mokutil --sb-state
mokutil reports secure boot enabled
=== Secure Boot State Check ===
Checker: UEFI (mokutil)
Current state: enabled
FAIL: Secure boot is not disabled (current state: SecureBootState.ENABLED)
------------------------------------------------------------------------- >8 ---
Outcome: job failed

Testing on Baoshan: UC 22 Image with Secure Boot Disabled (UBOOT + UKI image with UEFI)

  • To test sideload functionality, remove the manifest requirement (sideloading cannot test the manifest setting).
  • Run the command: checkbox-baoshan.checkbox-cli run com.canonical.certification::miscellanea/secure-boot-enabled com.canonical.certification::miscellanea/secure-boot-disabled
  • If the manifest works, the test for Verify secure boot is enabled when manifest indicates support will be skipped. It will run now since the manifest requirement is removed.
===========================[ Running Selected Jobs ]============================
==============[ Running job 1 / 2. Estimated time left: 0:00:01 ]===============
-------[ Verify secure boot is enabled when manifest indicates support ]--------
ID: com.canonical.certification::miscellanea/secure-boot-enabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Core via hostnamectl
Initialized UEFISecureBootChecker with hostfs_prefix='/var/lib/snapd/hostfs', ubuntu_variant='UbuntuVariant.CORE'
Running mokutil --sb-state
mokutil reports secure boot disabled
=== Secure Boot State Check ===
Checker: UEFI (mokutil)
Current state: disabled
FAIL: Secure boot is not enable (current state: SecureBootState.DISABLED)
------------------------------------------------------------------------- >8 ---
Outcome: job failed
==============[ Running job 2 / 2. Estimated time left: 0:00:00 ]===============
-----[ Verify secure boot is disabled when manifest indicates no support ]------
ID: com.canonical.certification::miscellanea/secure-boot-disabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Core via hostnamectl
Initialized UEFISecureBootChecker with hostfs_prefix='/var/lib/snapd/hostfs', ubuntu_variant='UbuntuVariant.CORE'
Running mokutil --sb-state
mokutil reports secure boot disabled
=== Secure Boot State Check ===
Checker: UEFI (mokutil)
Current state: disabled
PASS: Secure boot is disabled
------------------------------------------------------------------------- >8 ---
Outcome: job passed

Testing on Baoshan: Ubuntu 22 Image with Secure Boot Enabled

  • To test sideload functionality, remove the manifest requirement (sideloading cannot test the manifest setting).
  • Run the command: checkbox-baoshan.checkbox-cli run com.canonical.certification::miscellanea/secure-boot-enabled com.canonical.certification::miscellanea/secure-boot-disabled
  • If the manifest works, the test for Verify secure boot is disabled when manifest indicates no support will be skipped. It will run now since the manifest requirement is removed.
===========================[ Running Selected Jobs ]============================
==============[ Running job 1 / 2. Estimated time left: 0:00:01 ]===============
-------[ Verify secure boot is enabled when manifest indicates support ]--------
ID: com.canonical.certification::miscellanea/secure-boot-enabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Classic via hostnamectl
Initialized UEFISecureBootChecker with hostfs_prefix='/var/lib/snapd/hostfs', ubuntu_variant='UbuntuVariant.CLASSIC'
Running mokutil --sb-state
mokutil reports secure boot enabled
=== Secure Boot State Check ===
Checker: UEFI (mokutil)
Current state: enabled
PASS: Secure boot is enabled
------------------------------------------------------------------------- >8 ---
Outcome: job passed
==============[ Running job 2 / 2. Estimated time left: 0:00:00 ]===============
-----[ Verify secure boot is disabled when manifest indicates no support ]------
ID: com.canonical.certification::miscellanea/secure-boot-disabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Classic via hostnamectl
Initialized UEFISecureBootChecker with hostfs_prefix='/var/lib/snapd/hostfs', ubuntu_variant='UbuntuVariant.CLASSIC'
Running mokutil --sb-state
mokutil reports secure boot enabled
=== Secure Boot State Check ===
Checker: UEFI (mokutil)
Current state: enabled
FAIL: Secure boot is not disabled (current state: SecureBootState.ENABLED)
------------------------------------------------------------------------- >8 ---
Outcome: job failed

Copy link

codecov bot commented Jul 4, 2025

Codecov Report

Attention: Patch coverage is 98.77676% with 4 lines in your changes missing coverage. Please review.

Project coverage is 51.04%. Comparing base (2dab338) to head (4b08b40).
Report is 20 commits behind head on main.

Files with missing lines Patch % Lines
providers/base/bin/check_secure_boot_state.py 98.77% 1 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1998      +/-   ##
==========================================
+ Coverage   50.60%   51.04%   +0.43%     
==========================================
  Files         384      385       +1     
  Lines       41180    41553     +373     
  Branches     7636     7717      +81     
==========================================
+ Hits        20841    21210     +369     
+ Misses      19594    19593       -1     
- Partials      745      750       +5     
Flag Coverage Δ
provider-base 27.38% <98.77%> (+1.69%) ⬆️
provider-certification-client 57.14% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@seankingyang seankingyang changed the title Secure boot state checking for Classic and Core nomatter if the enable or disable Secure boot state checking for Classic and Core nomatter if the enable or disable (New) Jul 4, 2025
@seankingyang seankingyang marked this pull request as ready for review July 9, 2025 06:49
@rickwu666666
Copy link
Contributor

I've tried with my project which is ARM64/secure boot enable/FIT image/ubuntu core20. And everything works as expected.
pass result:

==============[ Running job 2 / 3. Estimated time left: 0:00:01 ]===============
-------[ Verify secure boot is enabled when manifest indicates support ]--------
ID: com.canonical.certification::miscellanea/secure-boot-enabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Core via hostnamectl
Initialized FITImageSecureBootChecker with hostfs_prefix='/var/lib/snapd/hostfs', ubuntu_variant='UbuntuVariant.CORE'
Detected Ubuntu variant: UbuntuVariant.CORE
Running in snap environment, using hostfs prefix: /var/lib/snapd/hostfs
No boot kernel found in common locations: ['/boot/vmlinuz', '/boot/vmlinuz-5.4.0-1079-shiner-devel', '/boot/kernel.img', '/boot/Image']
Search patterns (hostfs prioritized): ['/var/lib/snapd/hostfs/snap/*/current/kernel.img', '/var/lib/snapd/hostfs/snap/*/*/kernel.img', '/var/lib/snapd/hostfs/var/lib/snapd/seed/systems/*/kernel/kernel.img', '/var/lib/snapd/hostfs/run/mnt/ubuntu-boot/uboot/ubuntu/*/kernel.img', '/var/lib/snapd/hostfs/boot/uboot/*/kernel.img', '/snap/*/current/kernel.img', '/snap/*/*/kernel.img', '/var/lib/snapd/seed/systems/*/kernel/kernel.img', '/run/mnt/ubuntu-boot/uboot/ubuntu/*/kernel.img', '/boot/uboot/*/kernel.img']
Using snap kernel: /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img
Checking image: /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img
Checking signature of image: /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img
Image /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img is signed
Image /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img is signed
=== Secure Boot State Check ===
Checker: FIT Image (dumpimage)
Current state: enabled
PASS: Secure boot is enabled
------------------------------------------------------------------------- >8 ---
Outcome: job passed
==============[ Running job 3 / 3. Estimated time left: 0:00:00 ]===============
-----[ Verify secure boot is disabled when manifest indicates no support ]------
ID: com.canonical.certification::miscellanea/secure-boot-disabled
Category: com.canonical.plainbox::miscellanea
Job cannot be started because:
 - resource expression "manifest.has_secure_boot == 'False'" evaluates to false
Outcome: job cannot be started

fail result

==============[ Running job 2 / 3. Estimated time left: 0:00:01 ]===============
-------[ Verify secure boot is enabled when manifest indicates support ]--------
ID: com.canonical.certification::miscellanea/secure-boot-enabled
Category: com.canonical.plainbox::miscellanea
Job cannot be started because:
 - resource expression "manifest.has_secure_boot == 'True'" evaluates to false
Outcome: job cannot be started
==============[ Running job 3 / 3. Estimated time left: 0:00:00 ]===============
-----[ Verify secure boot is disabled when manifest indicates no support ]------
ID: com.canonical.certification::miscellanea/secure-boot-disabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Core via hostnamectl
Initialized FITImageSecureBootChecker with hostfs_prefix='/var/lib/snapd/hostfs', ubuntu_variant='UbuntuVariant.CORE'
Detected Ubuntu variant: UbuntuVariant.CORE
Running in snap environment, using hostfs prefix: /var/lib/snapd/hostfs
No boot kernel found in common locations: ['/boot/vmlinuz', '/boot/vmlinuz-5.4.0-1079-shiner-devel', '/boot/kernel.img', '/boot/Image']
Search patterns (hostfs prioritized): ['/var/lib/snapd/hostfs/snap/*/current/kernel.img', '/var/lib/snapd/hostfs/snap/*/*/kernel.img', '/var/lib/snapd/hostfs/var/lib/snapd/seed/systems/*/kernel/kernel.img', '/var/lib/snapd/hostfs/run/mnt/ubuntu-boot/uboot/ubuntu/*/kernel.img', '/var/lib/snapd/hostfs/boot/uboot/*/kernel.img', '/snap/*/current/kernel.img', '/snap/*/*/kernel.img', '/var/lib/snapd/seed/systems/*/kernel/kernel.img', '/run/mnt/ubuntu-boot/uboot/ubuntu/*/kernel.img', '/boot/uboot/*/kernel.img']
Using snap kernel: /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img
Checking image: /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img
Checking signature of image: /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img
Image /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img is signed
Image /var/lib/snapd/hostfs/snap/hbt-imx-kernel/current/kernel.img is signed
=== Secure Boot State Check ===
Checker: FIT Image (dumpimage)
Current state: enabled
FAIL: Secure boot is not disabled (current state: SecureBootState.ENABLED)
------------------------------------------------------------------------- >8 ---
Outcome: job failed

rickwu666666
rickwu666666 previously approved these changes Jul 10, 2025
Copy link
Contributor

@rickwu666666 rickwu666666 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@tomli380576 tomli380576 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm overall just small things that can help simplify enum usage

@zongminl zongminl changed the title Secure boot state checking for Classic and Core nomatter if the enable or disable (New) Secure boot state checking for Classic and Core no matter if the enable or disable (New) Jul 25, 2025
@rickwu666666
Copy link
Contributor

Tried with uc24/arm64/secure boot enabled platform. And test result were expected.

pass result

==============[ Running job 2 / 3. Estimated time left: 0:00:01 ]===============
-------[ Verify secure boot is enabled when manifest indicates support ]--------
ID: com.canonical.certification::miscellanea/secure-boot-enabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Core via hostnamectl
Initialized FITImageSecureBootChecker with hostfs_prefix='/var/lib/snapd/hostfs', ubuntu_variant='UbuntuVariant.CORE'
Detected Ubuntu variant: UbuntuVariant.CORE
Running in snap environment, using hostfs prefix: /var/lib/snapd/hostfs
No boot kernel found in common locations: ['/boot/vmlinuz', '/boot/vmlinuz-6.8.0-1007-renesas', '/boot/kernel.img', '/boot/Image']
Search patterns (hostfs prioritized): ['/var/lib/snapd/hostfs/snap/*/current/kernel.img', '/var/lib/snapd/hostfs/snap/*/*/kernel.img', '/var/lib/snapd/hostfs/var/lib/snapd/seed/systems/*/kernel/kernel.img', '/var/lib/snapd/hostfs/run/mnt/ubuntu-boot/uboot/ubuntu/*/kernel.img', '/var/lib/snapd/hostfs/boot/uboot/*/kernel.img', '/snap/*/current/kernel.img', '/snap/*/*/kernel.img', '/var/lib/snapd/seed/systems/*/kernel/kernel.img', '/run/mnt/ubuntu-boot/uboot/ubuntu/*/kernel.img', '/boot/uboot/*/kernel.img']
Using snap kernel: /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img
Checking image: /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img
Checking signature of image: /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img
Image /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img is signed
Image /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img is signed
=== Secure Boot State Check ===
Checker: FIT Image (dumpimage)
Current state: enabled
PASS: Secure boot is enabled
------------------------------------------------------------------------- >8 ---
Outcome: job passed
==============[ Running job 3 / 3. Estimated time left: 0:00:00 ]===============
-----[ Verify secure boot is disabled when manifest indicates no support ]------
ID: com.canonical.certification::miscellanea/secure-boot-disabled
Category: com.canonical.plainbox::miscellanea
Job cannot be started because:
 - resource expression "manifest.has_secure_boot == 'False'" evaluates to false
Outcome: job cannot be started

Failed result

==============[ Running job 2 / 3. Estimated time left: 0:00:01 ]===============
-------[ Verify secure boot is enabled when manifest indicates support ]--------
ID: com.canonical.certification::miscellanea/secure-boot-enabled
Category: com.canonical.plainbox::miscellanea
Job cannot be started because:
 - resource expression "manifest.has_secure_boot == 'True'" evaluates to false
Outcome: job cannot be started
==============[ Running job 3 / 3. Estimated time left: 0:00:00 ]===============
-----[ Verify secure boot is disabled when manifest indicates no support ]------
ID: com.canonical.certification::miscellanea/secure-boot-disabled
Category: com.canonical.plainbox::miscellanea
... 8< -------------------------------------------------------------------------
Detected Ubuntu Core via hostnamectl
Initialized FITImageSecureBootChecker with hostfs_prefix='/var/lib/snapd/hostfs', ubuntu_variant='UbuntuVariant.CORE'
Detected Ubuntu variant: UbuntuVariant.CORE
Running in snap environment, using hostfs prefix: /var/lib/snapd/hostfs
No boot kernel found in common locations: ['/boot/vmlinuz', '/boot/vmlinuz-6.8.0-1007-renesas', '/boot/kernel.img', '/boot/Image']
Search patterns (hostfs prioritized): ['/var/lib/snapd/hostfs/snap/*/current/kernel.img', '/var/lib/snapd/hostfs/snap/*/*/kernel.img', '/var/lib/snapd/hostfs/var/lib/snapd/seed/systems/*/kernel/kernel.img', '/var/lib/snapd/hostfs/run/mnt/ubuntu-boot/uboot/ubuntu/*/kernel.img', '/var/lib/snapd/hostfs/boot/uboot/*/kernel.img', '/snap/*/current/kernel.img', '/snap/*/*/kernel.img', '/var/lib/snapd/seed/systems/*/kernel/kernel.img', '/run/mnt/ubuntu-boot/uboot/ubuntu/*/kernel.img', '/boot/uboot/*/kernel.img']
Using snap kernel: /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img
Checking image: /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img
Checking signature of image: /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img
Image /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img is signed
Image /var/lib/snapd/hostfs/snap/renesas-kernel/current/kernel.img is signed
=== Secure Boot State Check ===
Checker: FIT Image (dumpimage)
Current state: enabled
FAIL: Secure boot is not disabled (current state: SecureBootState.ENABLED)
------------------------------------------------------------------------- >8 ---
Outcome: job failed

Copy link
Contributor

@rickwu666666 rickwu666666 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants