Skip to content

Conversation

@khaeru
Copy link
Member

@khaeru khaeru commented Apr 19, 2024

This PR adds:

  1. A CSV file with the current output of the legacy reporting, run on snapshot-1.
  2. A new test, .test_report.test_compare(), and associated utility function that:
    • Runs the genno-based reporting on the stored solution data from snapshot-1.
    • Compares the output to the stored output from (1).

This functions as a test of known differences between the output of the legacy and genno-based reporting for this particular scenario. The added variables NOT_IMPLEMENTED_MEASURE and NOT_IMPLEMENTED_IAMC can be modified and reduced as the genno-based reporting is expanded towards feature parity with the legacy reporting.

Also:

  • Refine handling of "iamc:" sections in the reporting config YAML file.
  • Extend the legacy reporting config file default_units.yaml to suppress some of the many warnings about missing "conversion factors".
  • Add .report.Config.modules and .iter_callbacks().
  • Add at key "all::iamc" a task that concatenates IAMC-structured inputs; this can be extended by various modules (e.g. as is done in .compat).

How to review

  • Read the changes in “What's new” and additions to the docs.
  • Skim the added test.
  • Note that CI checks all pass.

PR checklist

  • Continuous integration checks all ✅
  • Add or expand tests; coverage checks both ✅
  • Add, expand, or update documentation.
  • Update doc/whatsnew.

@khaeru khaeru added report genno-based reporting and post-solve processing report-legacy Legacy reporting labels Apr 19, 2024
@codecov
Copy link

codecov bot commented Apr 19, 2024

Codecov Report

❌ Patch coverage is 91.46341% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.7%. Comparing base (2bc42a3) to head (f943878).
⚠️ Report is 18 commits behind head on main.

Files with missing lines Patch % Lines
message_ix_models/tools/iamc.py 82.9% 7 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #178     +/-   ##
=======================================
- Coverage   75.3%   74.7%   -0.7%     
=======================================
  Files        274     274             
  Lines      22492   22545     +53     
=======================================
- Hits       16958   16856    -102     
- Misses      5534    5689    +155     
Files with missing lines Coverage Δ
message_ix_models/cli.py 93.4% <ø> (ø)
message_ix_models/report/__init__.py 92.3% <100.0%> (+0.4%) ⬆️
message_ix_models/report/compat.py 99.1% <100.0%> (-0.1%) ⬇️
message_ix_models/report/config.py 100.0% <100.0%> (ø)
message_ix_models/report/sim.py 98.5% <100.0%> (ø)
message_ix_models/tests/report/test_legacy.py 50.0% <ø> (-50.0%) ⬇️
message_ix_models/tests/test_report.py 100.0% <100.0%> (ø)
message_ix_models/tools/add_dac/__init__.py 67.4% <100.0%> (ø)
message_ix_models/util/_message_ix.py 100.0% <100.0%> (ø)
message_ix_models/tools/iamc.py 91.5% <82.9%> (-5.4%) ⬇️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@khaeru khaeru self-assigned this Apr 22, 2024
@khaeru khaeru force-pushed the enh/test-report branch 2 times, most recently from 9f91bdd to 7bc4db1 Compare August 6, 2024 12:48
@khaeru khaeru marked this pull request as ready for review November 4, 2025 07:51
@khaeru
Copy link
Member Author

khaeru commented Nov 4, 2025

Hi @macflo8 @Wegatriespython @macflo8 @junukitashepard —as mentioned yesterday, I've brought this PR up to date and got it ready to merge. I've requested a review mainly "FYI".

Later, as the scope of IAMC output from the genno reporting grows, I will show how to adjust the NOT_IMPLEMENTED_{MEASURE,IAMC} variables (essentially, remove or edit entries) such that the test still passes.

@yiyi1991 FYI those variables function as the checklist you mentioned in Slack.

Only one ✅ is needed to merge.

Copy link
Contributor

@macflo8 macflo8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this. Looks ready be merged!

@khaeru khaeru merged commit 94414a2 into main Nov 6, 2025
25 checks passed
@khaeru khaeru deleted the enh/test-report branch November 6, 2025 13:39
@khaeru khaeru added this to the 2025-12 milestone Nov 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

report genno-based reporting and post-solve processing report-legacy Legacy reporting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants