fpm fails at building projects with too many files

[davidpfister]

Description

The original issue appeared when building the project feq-parse with the following configuration

Version:     0.10.0, alpha
Program:     fpm(1)
Description: A Fortran package manager and build system
Home Page:   https://github.com/fortran-lang/fpm
License:     MIT
OS Type:     Windows

The test project has a quite large number of files (170), which results in a very long command line for the final linking. On Windows, command lines are limited in size and the present build exceeds the limit. This has been tested with cmd, msys2 and powershell without any difference.

FWIW, Visual Studio creates temporary files (e.g. "RSP1.rsp") with the content of the command line and ifort is invoked as ifort @".\RSP1.rsp

Expected Behaviour

The project should build on Windows without having to split the files to 2 or more subprojects.

Version of fpm

0.10.0, alpha

Platform and Architecture

Windows 10

Additional Information

The end of the build log is:

gfortran    -O3 -funroll-loops -Wimplicit-interface -fPIC -fmax-errors=1 -fcoarray=single -fimplicit-none -Werror=implicit-interface  build\gfortran_2654F75F5833692A\feq-parse\test_test.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_abs_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_acos_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_asin_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_atan_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_cos_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_division_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_gaussian3d_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_linear_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_linear_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_linear_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_linear_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_linear_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_linear_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_linear_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_linear_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log10_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_log_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_monadic_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_print_tokens.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_random_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sech_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sin_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_sqrt_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tanh_sfp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_r1fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_r1fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_r2fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_r2fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_r3fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_r3fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_r4fp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_r4fp64.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_sfp32.f90.o build\gfortran_2654F75F5833692A\feq-parse\test_tan_sfp64.f90.o build\gfortran_63E63B2712D76620\feq-parse\libfeq-parse.a -o build\gfortran_688A5DB7BAD2F9DA\test\test.exe
 + gfortran    -O3 -funroll-loops -Wimplicit-interface -fPIC -fmax-errors=1 -fcoarray=single -fimplicit-none -Werror=implicit-interface  build\gfortran_2654F75F5833692A\feq-parse\example_scalar_with_scalar_eval.f90.o build\gfortran_63E63B2712D76620\feq-parse\libfeq-parse.a -o build\gfortran_688A5DB7BAD2F9DA\example\scalar_with_scalar_eval.exe
The command line is too long.
<ERROR> Unable to find build log " test.exe.log "
<ERROR> Compilation failed for object " test.exe "

[perazz]

Windows has a hard limit of 32767 characters per command line that cannot be avoided.
I believe fpm cannot use something like rsp files, because the gfortran toolchains do not support them.

Instead, I think it would be viable to change the linking procedure according to the GCC guideline:

• add all objects to a static library, one by one
• link the static library so the linking command will always be short

This would be suitable for all compilers and platforms.

[davidpfister]

That looks like a good and quite simple idea to implement

[ivan-pi]

Are you sure the GCC toolchain doesn't support response files (the thread you linked is from 2009)? It appears in the man page: https://man7.org/linux/man-pages/man1/gcc.1.html

The full entry in the gcc man reads:

@file
           Read command-line options from file.  The options read are inserted in place of the
           original @file option.  If file does not exist, or cannot be read, then the option will
           be treated literally, and not removed.

           Options in file are separated by whitespace.  A whitespace character may be included in
           an option by surrounding the entire option in either single or double quotes.  Any
           character (including a backslash) may be included by prefixing the character to be
           included with a backslash.  The file may itself contain additional @file options; any
           such options will be processed recursively.

The NAG Fortran Compiler also has this option:

-indirect file
   Read the contents of file as additional arguments to the compiler driver. This 
   option may also be given by “@file”; note in this case there is no space between the 
   ‘@’ and the file name.

Also wanted to note that Ninja also uses response files on Windows for this purpose: https://ninja-build.org/manual.html (search for "rspfile")