t/ack-x.t fails on NetBSD-8

[jkeenan]

Today I tried to install App::Ack on a NetBSD-8.0 VM managed by Vagrant. I tried three different ways and got the same test failure each way.

The three ways:

• Include in a SHELL script inside the Vagrantfile: sudo pkgin install p5-ack
• Include in a SHELL script inside the Vagrantfile: sudo cpan App::Ack
• Enter the VM and call: sudo cpan App::Ack

Test failure output from the 3rd attempt:

$ sudo cpan App::Ack
Loading internal logger. Log::Log4perl recommended for better logging
Reading '/root/.cpan/Metadata'
  Database was generated on Fri, 23 Aug 2019 22:29:03 GMT
Running install for module 'App::Ack'
Checksum for /root/.cpan/sources/authors/id/P/PE/PETDANCE/ack-v3.1.0.tar.gz ok
'YAML' not installed, will not store persistent state
Configuring P/PE/PETDANCE/ack-v3.1.0.tar.gz with Makefile.PL
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for ack
Writing MYMETA.yml and MYMETA.json
  PETDANCE/ack-v3.1.0.tar.gz
  /usr/pkg/bin/perl Makefile.PL -- OK
Running make for P/PE/PETDANCE/ack-v3.1.0.tar.gz
cp lib/App/Ack/Filter/IsGroup.pm blib/lib/App/Ack/Filter/IsGroup.pm
cp lib/App/Ack/Filter/Default.pm blib/lib/App/Ack/Filter/Default.pm
cp lib/App/Ack/Filter/Collection.pm blib/lib/App/Ack/Filter/Collection.pm
cp lib/App/Ack/Filter/Match.pm blib/lib/App/Ack/Filter/Match.pm
cp lib/App/Ack/Filter/IsPathGroup.pm blib/lib/App/Ack/Filter/IsPathGroup.pm
cp lib/App/Ack/ConfigFinder.pm blib/lib/App/Ack/ConfigFinder.pm
cp lib/App/Ack/ConfigDefault.pm blib/lib/App/Ack/ConfigDefault.pm
cp lib/App/Ack/Filter/MatchGroup.pm blib/lib/App/Ack/Filter/MatchGroup.pm
cp lib/App/Ack/Filter/FirstLineMatch.pm blib/lib/App/Ack/Filter/FirstLineMatch.pm
cp lib/App/Ack/ConfigLoader.pm blib/lib/App/Ack/ConfigLoader.pm
cp lib/App/Ack.pm blib/lib/App/Ack.pm
cp lib/App/Ack/Filter/Is.pm blib/lib/App/Ack/Filter/Is.pm
cp lib/App/Ack/File.pm blib/lib/App/Ack/File.pm
cp lib/App/Ack/Filter/Extension.pm blib/lib/App/Ack/Filter/Extension.pm
cp lib/App/Ack/Filter/ExtensionGroup.pm blib/lib/App/Ack/Filter/ExtensionGroup.pm
cp lib/App/Ack/Filter/Inverse.pm blib/lib/App/Ack/Filter/Inverse.pm
cp lib/App/Ack/Filter.pm blib/lib/App/Ack/Filter.pm
cp lib/App/Ack/Files.pm blib/lib/App/Ack/Files.pm
cp lib/App/Ack/Filter/IsPath.pm blib/lib/App/Ack/Filter/IsPath.pm
cp ack blib/script/ack
"/usr/pkg/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/ack
Manifying 1 pod document
  PETDANCE/ack-v3.1.0.tar.gz
  /usr/bin/make -- OK
Running make test
Manifying 1 pod document
"/usr/pkg/bin/perl" t/runtests.pl 0 0 "blib/lib" "blib/arch" t/*.t
Running tests on blib/script/ack, ACK_TEST_STANDALONE=0
t/00-load.t .................. 1/20 # Testing ack v3.1.0 under Perl v5.28.2, /usr/pkg/bin/perl
# Using File::Next 1.16
# Using File::Spec 3.74
# Using Getopt::Long 2.5
# Using Pod::Perldoc 3.2801
# Using Pod::Text 4.10
# Using Pod::Usage 1.69
# Using Test::Harness 3.42
# Using Test::More 1.302167
# PATH=/usr/sbin/:/usr/pkg/bin/:/usr/bin:/bin:/usr/pkg/bin:/usr/local/bin
t/00-load.t .................. ok     
t/ack-1.t .................... ok     
...
t/ack-w.t .................... ok     
t/ack-x.t .................... 1/2     # No tests run!

#   Failed test 'No tests run for subtest "GH #175"'
#   at t/ack-x.t line 106.
File /tmp/XkYaQHD9Yb is still readable despite our attempts to changes its permissions at t/ack-x.t line 79.
# Looks like your test exited with 255 just after 2.
t/ack-x.t .................... Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 1/2 subtests 
t/anchored.t ................. ok   
t/asp-net-ext.t .............. ok   
...
Test Summary Report
-------------------
t/ack-x.t                  (Wstat: 65280 Tests: 2 Failed: 1)
  Failed test:  2
  Non-zero exit status: 255
Files=82, Tests=933, 78 wallclock secs ( 0.19 usr  0.29 sys + 55.97 cusr 22.31 csys = 78.76 CPU)
Result: FAIL
Failed 1/82 test programs. 1/933 subtests failed.
*** Error code 255

Stop.
make: stopped in /root/.cpan/build/ack-v3.1.0-1
  PETDANCE/ack-v3.1.0.tar.gz
  /usr/bin/make test -- NOT OK

The vendor perl is 5.28.2 threaded.

Summary of my perl5 (revision 5 version 28 subversion 2) configuration:
   
  Platform:
    osname=netbsd
    osvers=8.0
    archname=x86_64-netbsd-thread-multi
    uname='netbsd amd64-nb8.netbsd.org 8.0 netbsd 8.0 (libkver) #0: tue jan 19 00:00:00 utc 2038 root@localhost:sysarchamd64compilelibkver amd64 '
    config_args='-sde -Dldflags=   -pthread  -L/usr/lib -Wl,-R/usr/lib -Wl,-R/usr/pkg/lib -Duseshrplib -Duseithreads -Uusemymalloc'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='gcc'
    ccflags ='-O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -fwrapv -fno-strict-aliasing -pipe'
    optimize='-O2 -D_FORTIFY_SOURCE=2  -pthread  -I/usr/include'
    cppflags='-O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -fwrapv -fno-strict-aliasing -pipe'
    ccversion=''
    gccversion='5.5.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='gcc'
    ldflags =' -pthread -L/usr/lib -Wl,-R/usr/lib -Wl,-R/usr/pkg/lib -L/usr/pkg/lib'
    libpth=/lib /usr/lib /usr/pkg/lib
    libs=-lm -lcrypt -lpthread
    perllibs=-lm -lcrypt -lpthread
    libc=/lib/libc.so
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E  -Wl,-R/usr/pkg/lib/perl5/5.28.0/x86_64-netbsd-thread-multi/CORE'
    cccdlflags='-DPIC -fPIC '
    lddlflags='-shared  -L/usr/lib -Wl,-R/usr/lib -Wl,-R/usr/pkg/lib -L/usr/pkg/lib'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under netbsd
  Compiled at Jul  1 2019 02:46:37
  @INC:
    lib
    /usr/pkg/lib/perl5/site_perl/5.28.0/x86_64-netbsd-thread-multi
    /usr/pkg/lib/perl5/site_perl/5.28.0
    /usr/pkg/lib/perl5/vendor_perl/5.28.0/x86_64-netbsd-thread-multi
    /usr/pkg/lib/perl5/vendor_perl/5.28.0
    /usr/pkg/lib/perl5/5.28.0/x86_64-netbsd-thread-multi
    /usr/pkg/lib/perl5/5.28.0

[n1vux]

This is is interesting.

HOW is NetBSD (user-space or kernel) and/or Vagrant preventing the .t file from chmod'ing a file to inaccessible? (and WHY ??)

It may be some nanny policy to prevent alienating an asset?

User workaround for James @jkeenan would be to use the force option on CPAN install. I note the install self-check only failed one test. Since the Vagrant environment won't let us create a file ack can't read, not knowing if ack will properly fail there on a file it can't read is harmless 😄 so forced install is safe, and this is the expected use-case of the forced-install: when the failed tests are don't-care, so go ahead and do it.

Our minimum response could be

• add DOC saying "some VMs will prevent some tests for passing if the e.g. prevent a program from chmoding a file such that the owner/creator can't read it; if only a one or a handful of such corner cases fail, go ahead and do cpan(or cpanm) force install.
• and add similar "your environment is weird that this test's setup fails; if this is the only test failing, feel safe in forcing install" to error message File /tmp/XkYaQHD9Yb is still readable despite our attempts to changes its permissions

Further Distribution Workaround (if we care to do more than above) would be making the chmod-dependent subtest "GH #175"' at t/ack-x.t line 106 an Author-only test, and/or replace the bailout error with SKIP.
(I say and/or because I consider the possibility of duplicating this test into Author-only section so it's a hard-fail for GitHub CI and core developers prepping a release or running author-tests by request, but a fail-to-SKIP for folks installing from CPAN, providing a soft (if they read the details) warning that their environment is a trifle strange. OTOH I like failing with recommendation to force better than either of these alternatives.)

Xref - the test failing was added in response to GH #175 according to subtest name