Rework Fortran configure to allow cross compile overrides #5720

byrnHDF · 2025-08-11T20:42:59Z

Another possibility using defaults and toolchain file could overwrite.
See #5709 and #5718

The expectation is that cross-compiling will use a toolchain file to provide the correct values.
The defaults are only one way to provide defaults for a specific compiler.

NOTE: The supplied toolchain and Fortran pre-gen files are specific for mingw/gfortran - each platform/compiler will need to supply its own set of files/values.

Important

Rework HDF5UseFortran.cmake to support cross-compilation by providing default KINDs and sizes.

Behavior:
- Modify HDF5UseFortran.cmake to handle cross-compilation by providing default KINDs and sizes when CMAKE_CROSSCOMPILING is true.
- Default values for INTEGER and REAL KINDs and their sizes are set when cross-compiling.
- Error messages are adjusted to reflect cross-compilation scenarios.
Logic:
- Conditional checks added for CMAKE_CROSSCOMPILING to determine if default values should be used.
- Default values include INTEGER KINDs {1,2,4,8,16} and REAL KINDs {4,8,10,16}.
- Default sizes for native kinds are set to 4 for INTEGER and REAL, and 8 for DOUBLE PRECISION when cross-compiling.
Misc:
- Adjustments to verbose messages to indicate when defaults are used during cross-compilation.

^{This description was created by}^{for 0b8f640. You can customize this summary. It will automatically update as commits are pushed.}

ellipsis-dev · 2025-08-11T20:46:22Z

config/HDF5UseFortran.cmake

+  set (PAC_FC_ALL_INTEGER_KINDS "\{1,2,4,8,16\}" CACHE INTERNAL "Find available INTEGER KINDs for Fortran")
+  set (PAC_FC_ALL_REAL_KINDS "\{4,8,10,16\}" CACHE INTERNAL "Find available REAL KINDs for Fortran")
+  set (${HDF_PREFIX}_PAC_FC_MAX_REAL_PRECISION 33 CACHE INTERNAL "Maximum decimal precision for REALs in Fortran")
+  set (PAC_FORTRAN_NUM_INTEGER_KINDS "5" CACHE INTERNAL "Number of valid integer kinds for Fortran")


In the cross‐compile default branch, the default integer kinds count is inconsistent: 'PAC_FORTRAN_NUM_INTEGER_KINDS' is set to 5, but H5CONFIG_F_NUM_IKIND is hard‐coded to 4. Please align these values.

These are two different items.

ellipsis-dev · 2025-08-11T20:46:22Z

config/HDF5UseFortran.cmake

+else ()
+  # Use the default if there's no cache variable and cross-compiling
+  message (VERBOSE "Find available REAL KINDs for Fortran... yes (cross-compile default)")
+  set (PAC_FC_ALL_REAL_KINDS_SIZEOF "\{4,8,16,16\}" CACHE INTERNAL "Find available REAL KINDs for Fortran")


The default REAL KIND SIZEOF value contains a duplicate '16' (i.e. "{4,8,16,16}"). Verify if this duplication is intentional or if it should differ (e.g. perhaps "{4,8,10,16}").

Suggested change

set (PAC_FC_ALL_REAL_KINDS_SIZEOF "\{4,8,16,16\}" CACHE INTERNAL "Find available REAL KINDs for Fortran")

set (PAC_FC_ALL_REAL_KINDS_SIZEOF "\{4,8,10,16\}" CACHE INTERNAL "Find available REAL KINDs for Fortran")

These are correct values for linux

It's not Linux that determines this, but the compiler. Interpretation of KIND values depends on the compiler. For example, NAG, by default, uses sequential kind numbering, which is not what GNU does.

Yes, the idea is that toolchain specific overrides will modify this. This is just a simple default - the other option will be to skip the default and require an override.

I could also change the "(cross-compile default)" to "(cross-compile gfortran default)"

byrnHDF · 2025-08-12T20:02:30Z

The mingw dross-compile gets past the configure stage and then chokes on:
FAILED: [code=1] fortran/src/CMakeFiles/H5_buildiface.dir/H5_buildiface.F90.obj
FAILED: [code=1] fortran/src/CMakeFiles/H5match_types.dir/H5match_types.c.obj

because the values from the defaults are not correctly formatted. Once the syntax is correct then we can update the toolchain with the correct values.

byrnHDF · 2025-09-02T22:55:28Z

The latest update fixes the Fortran configuration issue with PRE-GENERATED files. And the build succeeds. The tests fail because they are failing with the WINE emulator - maybe someone can suggest a fix?

byrnHDF · 2025-09-03T20:21:58Z

CMake applies the CMAKE_CROSSCOMPILING_EMULATOR for cross compiling and does not need to be specified in the command line. The WINEPATH ENV must be added to calls involving wine emulator. Added this to the mingw toolchain. Only 24 tests failed mostly h5_test_api and hl_test tests. Will need to investigate which of these tests are expected to fail with wine emulator.

byrnHDF · 2025-09-03T21:03:33Z

Fixed the h5_api_test by excluding them if cross-compiling.
Fixed the properties for HL and SWMR check
Leaving 5 failures:
3 - H5TEST-testhdf5-base (Failed)
8 - H5TESTXPR-cache_api (Failed)
65 - H5TEST-swmr (Failed)
89 - H5TEST-filenotclosed (Failed)
93 - H5TEST-err_compat (Failed)

byrnHDF · 2025-09-04T15:23:53Z

Note that 3 - H5TEST-testhdf5-base (Failed) is because :
HDF_CHECK_TYPE_SIZE (off_t ${HDF_PREFIX}_SIZEOF_OFF_T) needs to be 4 and not 8

byrnHDF · 2025-09-04T19:11:16Z

Although the CI doesn't show it - there are only 4 failing tests on my local fedora system. The ubuntu runner seems to have a char set issue causing problems. Likely a missing system dependency?
As far as code changes for CMake, I believe this PR to be complete.

hmaarrfk · 2025-09-04T19:35:08Z

16k lines!!!! I really hope this is "improving things" beyond just "cross compilation workflows". I really think cross compilation is critical, but this has definitely gone "beyond" the scope I originally imaged this being.

byrnHDF · 2025-09-04T20:18:49Z

So the big change is that CMake will apply the CMAKE_CROSS_COMPILING_EMULATER as needed - had to remove that from the commands
The wine64 emulator requires setting WINEPATH env var. A global setting didn't work for me so I added that to the test properties.
Finally the other changes concern the Fortran configure and the Fortran generated files.

hmaarrfk · 2025-09-04T23:54:28Z

Are you able to provide points and tips to the failures in: conda-forge/hdf5-feedstock#268

byrnHDF · 2025-09-05T12:34:37Z

Are you able to provide points and tips to the failures in: conda-forge/hdf5-feedstock#268

I haven't tried mpi and cross-compiling - is that the issue?

byrnHDF · 2025-09-05T12:45:22Z

Note that the supplied pre-gen files and configure defaults were specific for mingw-gfortran. Other compilers will likely need other values and files. The best thing would be to create a stable of toolchain files and dirs.

hmaarrfk · 2025-09-06T21:09:36Z

i'm honestly not sure how much time i have to troubleshoot this with you all. i'm really sorry. you can see my checklist, i got pretty far. but i need some hands on keyboard debugging from some1 who is willing to look at things from a conda-forge standpoint. Its free to make a PR, which will run on our infrastructure.

To be honest, it is very likely that we will be able to, by 2026, to run on native runners for OSX on conda-forge. I'm willing to wait, or simply to stick on the HDF5 V1, and "not accelerate" its deployment.

byrnHDF · 2025-09-07T14:14:32Z

Yes I believe that running on native machines is the best plan. Our use of native runners has been very beneficial.

ajelenak · 2025-09-08T19:40:46Z

it is very likely that we will be able to, by 2026, to run on native runners for OSX on conda-forge.

@hmaarrfk Do you know when in 2026 this may happen?

hmaarrfk · 2025-09-08T21:15:50Z

conda-forge/conda-forge.github.io#1781 (comment)

Rework Fortran configure to allow cross compile overrides

0b8f640

byrnHDF self-assigned this Aug 11, 2025

byrnHDF requested review from lrknox, derobins, fortnern and jhendersonHDF as code owners August 11, 2025 20:43

github-project-automation bot added this to HDF5 - TRIAGE & TRACK Aug 11, 2025

byrnHDF requested review from qkoziol, vchoi-hdfgroup, bmribler, glennsong09, mattjala and brtnfld as code owners August 11, 2025 20:43

github-project-automation bot moved this to To be triaged in HDF5 - TRIAGE & TRACK Aug 11, 2025

byrnHDF mentioned this pull request Aug 11, 2025

Fix cross compiling without emulation #5718

Draft

5 tasks

ellipsis-dev bot reviewed Aug 11, 2025

View reviewed changes

byrnHDF and others added 13 commits August 12, 2025 07:30

Merge branch 'HDFGroup:develop' into develop-fortran-config

e217e9e

Add cross-compile template toolchain and use gfortran in defaults

5e01b7e

Add cross-compile action

88a5c69

install mingw

1d8d13a

correct yaml name

4c34d6b

Fix ext name

84aafb4

disable plugins until fixed

e10dc8d

correct syntax

c528c12

Try setup-mingw

b590e5c

Try a different toolchain with paths

d871ee0

fix toolchain

b0f0efb

Try explict install of gfortran

239db8e

Disable java

8fe8f0b

byrnHDF added 2 commits September 2, 2025 17:28

Update workflow with path to files

8f03e94

Use full path

19bff07

CMAKE_CROSSCOMPILING_EMULATOR is handled by CMake added PATH var

6773bc7

byrnHDF added 3 commits September 3, 2025 15:37

Correct no cross-compile Fortan custom_commands

2e626c3

Fix failing tests for mingw

aaddeb9

Correct location for property

acb0897

byrnHDF and others added 4 commits September 3, 2025 16:12

Disable HDF5_TEST_API option for cross-compile workflow

051a1af

Fix command properties

9179ab6

Merge develop and fix conflicts

79a83d3

Merge branch 'develop' into develop-fortran-config

68ee2b5

byrnHDF and others added 4 commits September 4, 2025 10:40

Fix the HDF5_SIZEOF_OFF_T issue with mingw

487769c

Merge branch 'HDFGroup:develop' into develop-fortran-config

922b254

Correct OFF_T size for mingw

6c1e774

Allow cross-compile workflow to pass with failure testing

b9a5400

Merge branch 'HDFGroup:develop' into develop-fortran-config

de7ea6c

Merge branch 'HDFGroup:develop' into develop-fortran-config

2a41d84

	set (PAC_FC_ALL_REAL_KINDS_SIZEOF "\{4,8,16,16\}" CACHE INTERNAL "Find available REAL KINDs for Fortran")
	set (PAC_FC_ALL_REAL_KINDS_SIZEOF "\{4,8,10,16\}" CACHE INTERNAL "Find available REAL KINDs for Fortran")

Uh oh!

Rework Fortran configure to allow cross compile overrides #5720

Are you sure you want to change the base?

Rework Fortran configure to allow cross compile overrides #5720

Uh oh!

Conversation

byrnHDF commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ellipsis-dev bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

byrnHDF Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

byrnHDF Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

brtnfld Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

byrnHDF Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

byrnHDF Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

byrnHDF commented Aug 12, 2025

Uh oh!

byrnHDF commented Sep 2, 2025

Uh oh!

byrnHDF commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

byrnHDF commented Sep 3, 2025

Uh oh!

byrnHDF commented Sep 4, 2025

Uh oh!

byrnHDF commented Sep 4, 2025

Uh oh!

hmaarrfk commented Sep 4, 2025

Uh oh!

byrnHDF commented Sep 4, 2025

Uh oh!

hmaarrfk commented Sep 4, 2025

Uh oh!

byrnHDF commented Sep 5, 2025

Uh oh!

byrnHDF commented Sep 5, 2025

Uh oh!

hmaarrfk commented Sep 6, 2025

Uh oh!

byrnHDF commented Sep 7, 2025

Uh oh!

ajelenak commented Sep 8, 2025

Uh oh!

hmaarrfk commented Sep 8, 2025

Uh oh!

Uh oh!

byrnHDF commented Aug 11, 2025 •

edited

Loading

byrnHDF commented Sep 3, 2025 •

edited

Loading