Automate wake-on-LAN tests (New) #1819

eugene-yujinwu · 2025-03-24T11:11:50Z

Description

This is a clone PR of #1686.
The 1686 was not reviewed by Cert team, and it was reverted and this PR was re-submitted for review.
Please refer more detailed info in the #1686.
Thank you.

We would like to automate the wake-on-LAN test instead of running manual tests for it.
Please refer to wake-on-LAN-automatic-tests.md in the commit for more details

Resolved issues

Make the Wake-on-LAN test from manual to automatic.

Documentation

https://github.com/canonical/checkbox/pull/1819/files#diff-cb60a2a6986f9c704b5a299f24f29bbfc64064adc94aec85d4fa75c167ccbb01

Tests

"[wol_S3_auto] Both success": https://certification.canonical.com/submissions/status/300209

"[wol_S3_auto] Cannot connect to the WOL server" : https://certification.canonical.com/submissions/status/300211

"[wol_S3_auto] One of the WOL disabled" : https://certification.canonical.com/submissions/status/300210

"[wol_auto_test] One of NIC has wake-on-LAN issue" : https://certification.canonical.com/submissions/status/300212

"[wake-on-Lan] test for one NIC": https://certification.canonical.com/hardware/202406-34088/submission/408713/

codecov · 2025-03-24T11:12:43Z

Codecov Report

❌ Patch coverage is 95.45455% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.53%. Comparing base (560ff50) to head (a63674d).

Files with missing lines	Patch %	Lines
providers/base/bin/wol_client.py	93.02%	8 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1819      +/-   ##
==========================================
+ Coverage   52.33%   52.53%   +0.20%     
==========================================
  Files         392      394       +2     
  Lines       42070    42268     +198     
  Branches     7796     7824      +28     
==========================================
+ Hits        22018    22207     +189     
- Misses      19275    19283       +8     
- Partials      777      778       +1

Flag	Coverage Δ
provider-base	`29.33% <95.45%> (+0.81%)`	⬆️
provider-certification-client	`57.14% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

stanley31huang · 2025-06-30T10:40:06Z

@eugene-yujinwu
is it possible to provide how user could get the wol_server.py scripts in the README? as we need it be run in another client to make this test works.

eugene-yujinwu · 2025-07-01T09:26:35Z

@eugene-yujinwu is it possible to provide how user could get the wol_server.py scripts in the README? as we need it be run in another client to make this test works.

@stanley31huang A link to the wol_server.py was added. Thank you.

stanley31huang

LGTM

pieqq

I'm sorry this review took so long! It looks like a sensible change, and the different examples you provided from C3 are really helpful to see it in action!

I left a few comments below, please check them out and let me know if you have questions.

providers/base/units/ethernet/wake-on-LAN-automatic-tests.md

providers/base/bin/wol_check.py

pieqq · 2025-09-04T19:14:49Z

providers/base/bin/wol_client.py

+    req = urllib.request.Request(url, data=data_encoded, headers=headers)
+
+    attempts = 0
+    while attempts < retry:


In checkbox_support, we have a retry helper that can be used to re-run a given function a certain number of times. See this example for how to use it, and check if it could be a good use case for here.

Using a retry decorator appears more elegant, but the current approach is easier to understand and read, and it doesn’t require modifying the unit tests. If it be deemed necessary, I will make the corresponding changes.

In checkbox_support, we have a retry helper that can be used to re-run a given function a certain number of times. See this example for how to use it, and check if it could be a good use case for here.

For us, it's easier to maintain if new tests use the same retry mechanism, so please use it for your test. For the unit test, you can use mock_retry which was implemented to make it easier to write unit tests for scripts that make use of @retry. Please check test_eth_hotplugging.py for instance.

providers/base/units/ethernet/jobs.pxu

providers/base/bin/wol_client.py

providers/base/units/ethernet/manifest.pxu

pieqq

Thanks for the changes, I added a few more comments.

pieqq · 2025-09-19T07:23:34Z

providers/base/bin/wol_client.py

+    req = urllib.request.Request(url, data=data_encoded, headers=headers)
+
+    attempts = 0
+    while attempts < retry:


For us, it's easier to maintain if new tests use the same retry mechanism, so please use it for your test. For the unit test, you can use mock_retry which was implemented to make it easier to write unit tests for scripts that make use of @retry. Please check test_eth_hotplugging.py for instance.

providers/base/bin/wol_client.py

providers/base/units/ethernet/wake-on-LAN-automatic-tests.md

please refer to wake-on-LAN-automatic-tests.md for more details Co-authored-by: hanhsuan <[email protected]>

… checkbox. This is because it's not directly called by checkbox and introduces an unnecessary dependency on FastAPI.

…ues on some older python versions. Fix some issues per Stanley's review Add the related unit tests after code changes

Move inline function out of get_ip_mac per Stanley's comments

use more specific 4xx or 5xx status code from wol_server Co-authored-by: stanley31huang <[email protected]>

Check the return value when get the timestamp to prevent unexpected exception Update related unit tests

eugene-yujinwu · 2025-09-28T09:44:06Z

@pieqq Thank you for the review. New commit was pushed.

eugene-yujinwu changed the title ~~Automate wake-on-LAN tests (New)~~ Automate wake-on-LAN tests (New) Mar 24, 2025

Hook25 assigned pieqq Apr 18, 2025

stanley31huang previously approved these changes Jul 11, 2025

View reviewed changes

pieqq requested changes Sep 4, 2025

View reviewed changes

eugene-yujinwu dismissed stanley31huang’s stale review via 5cdca29 September 10, 2025 10:53

eugene-yujinwu force-pushed the automate_wake-on-lan branch 3 times, most recently from f0ffa6a to fa5a04e Compare September 10, 2025 11:34

eugene-yujinwu requested a review from pieqq September 11, 2025 03:56

pieqq requested changes Sep 19, 2025

View reviewed changes

eugene-yujinwu and others added 12 commits September 28, 2025 16:33

Automated the wake-on-LAN tests (New)

202e29a

please refer to wake-on-LAN-automatic-tests.md for more details Co-authored-by: hanhsuan <[email protected]>

Fix the unit test issue

b82806f

Remove wol_server.py and place it in a separate repository outside of…

9eec392

… checkbox. This is because it's not directly called by checkbox and introduces an unnecessary dependency on FastAPI.

remove f-string syntax not be supported in Python 3.5

4bab43d

Use socket and urllib to replace netifaces and urllib3 since some iss…

7781533

…ues on some older python versions. Fix some issues per Stanley's review Add the related unit tests after code changes

Add a has_wake_on_lan_server manifest per Stanley's comments

99b1396

Move inline function out of get_ip_mac per Stanley's comments

fix some of the new reviews from Stanley

c56c632

remove unused parameters in wol_check.py

65e4295

use more specific 4xx or 5xx status code from wol_server Co-authored-by: stanley31huang <[email protected]>

Use the /proc/stat to get the system boot time

88e2f63

Check the return value when get the timestamp to prevent unexpected exception Update related unit tests

fix some errors in the Readme

de65481

Add link to wol_server.py in README

ce9d2b8

Fix some issues per Pierre's comments

924ecda

eugene-yujinwu force-pushed the automate_wake-on-lan branch 2 times, most recently from f0bb3ed to 43ea23c Compare September 28, 2025 09:12

Use retry decorator per Pierre's suggestion

a63674d

eugene-yujinwu force-pushed the automate_wake-on-lan branch from 43ea23c to a63674d Compare September 28, 2025 09:27

eugene-yujinwu requested a review from pieqq September 28, 2025 09:43

Automate wake-on-LAN tests (New) #1819

Are you sure you want to change the base?

Automate wake-on-LAN tests (New) #1819

Uh oh!

Conversation

eugene-yujinwu commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Resolved issues

Documentation

Tests

Uh oh!

codecov bot commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

stanley31huang commented Jun 30, 2025

Uh oh!

eugene-yujinwu commented Jul 1, 2025

Uh oh!

stanley31huang left a comment

Choose a reason for hiding this comment

Uh oh!

pieqq left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pieqq Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

eugene-yujinwu Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

pieqq Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pieqq left a comment

Choose a reason for hiding this comment

Uh oh!

pieqq Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

eugene-yujinwu commented Sep 28, 2025

Uh oh!

Uh oh!

eugene-yujinwu commented Mar 24, 2025 •

edited

Loading

codecov bot commented Mar 24, 2025 •

edited

Loading