niimath not working

[ldaumail]

Hello,

I am trying to create an apptainer image for pyAFQ #103 to work on gpus using the following config .def file:

Bootstrap: docker
From: nvidia/cuda:12.1.0-devel-ubuntu22.04

%environment
    export DEBIAN_FRONTEND=noninteractive
    export PATH=/opt/anaconda/envs/afq/bin:/opt/anaconda/bin:${PATH}
    export LD_LIBRARY_PATH=/opt/anaconda/lib:${LD_LIBRARY_PATH}
    export PYTHONPATH=/opt/anaconda/envs/afq/lib/python3.11/site-packages:${PYTHONPATH}

%post
    # System update and basic tools installation
    apt-get update && apt-get install --assume-yes apt-transport-https \
        ca-certificates gnupg software-properties-common \
        gcc git wget curl numactl cmake 


    # Miniconda3
    curl -L "https://repo.anaconda.com/miniconda/Miniconda3-py311_25.1.1-1-Linux-x86_64.sh" \
        -o "/tmp/Miniconda3.sh"
    bash /tmp/Miniconda3.sh -b -p /opt/anaconda
    rm -rf /tmp/Miniconda3.sh
    cd /opt && eval "$(/opt/anaconda/bin/conda shell.bash hook)"
  
    # Create writable conda env for AFQ + niimath
    /opt/anaconda/bin/conda create -y -n afq python=3.11

    # Use the env’s python for installs
    /opt/anaconda/envs/afq/bin/pip install numpy>=2.0.0 scipy>=1.13.0 cython nibabel dipy tqdm fslpy
    /opt/anaconda/envs/afq/bin/pip install git+https://github.com/dipy/GPUStreamlines.git
    /opt/anaconda/envs/afq/bin/pip install git+https://github.com/36000/pyAFQ.git@dam
    
    #Address read only to executable permissions
    chmod +x /opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/bin/niimath || true    
    ln -sf /opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/bin/niimath /usr/local/bin/niimath || true    
    
    # Sanity check (Brainchop's niimath should now work in env)
    /opt/anaconda/envs/afq/bin/niimath --help || true
%runscript
    exec /opt/anaconda/envs/afq/bin/python "$@"

I also tried adding:

chmod -R a+rX /opt/anaconda
chmod a+rx /usr/local/bin/niimath
chmod a+rx /opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/bin/niimath

At the end of the %post section.

The error I get is:

INFO:AFQ:No stop mask given, using ACT and 3T prob maps esimated from T1w INFO:AFQ:Calculating _desc-bundles_tractography INFO:AFQ:Calculating _desc-T1w_mask.nii.gz ERROR:AFQ:Command failed: ['niimath', '/tmp/tmp5m0g1i1o/t1.nii.gz', '-conform', '-gz', '0', '-', '-odt', 'char'] ERROR:AFQ:Return code: 1 ERROR:AFQ:STDERR: Traceback (most recent call last): File "/opt/anaconda/envs/afq/bin/niimath", line 3, in <module> from niimath import main File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/__init__.py", line 38, in <module> os.chmod(bin, os.stat(bin).st_mode | 0o111) OSError: [Errno 30] Read-only file system: '/opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/bin/niimath' Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 884, in <lambda> lazy(lambda c,args: f(inputtup, calctup, c, args), c, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 873, in _call_calc r = c.eager_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 448, in eager_call res = self.function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/tasks/decorators.py", line 84, in wrapper_as_file gen, meta = func( ^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/tasks/data.py", line 1426, in t1_brain_mask return run_brainchop(nib.load(t1_file), "mindgrab"), dict( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/nn/brainchop.py", line 68, in run_brainchop volume, header = _run_brainchop_command(conform, [tmp_t1_file]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/nn/brainchop.py", line 36, in _run_brainchop_command return func(*args) ^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/brainchop/niimath/__init__.py", line 273, in conform res = subprocess.run(cmd, capture_output=True, check=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['niimath', '/tmp/tmp5m0g1i1o/t1.nii.gz', '-conform', '-gz', '0', '-', '-odt', 'char']' returned non-zero exit status 1. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 884, in <lambda> lazy(lambda c,args: f(inputtup, calctup, c, args), c, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 868, in _call_calc for (p,arg) in zip(c.signature.parameters.values(), argvals): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 855, in _source_lookup val = lazycalc()[oidx] ^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to <lambda>(<immlib.calc object at 0x1555009b85c0>, (6, 7, 8, 9)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 884, in <lambda> lazy(lambda c,args: f(inputtup, calctup, c, args), c, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 868, in _call_calc for (p,arg) in zip(c.signature.parameters.values(), argvals): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 855, in _source_lookup val = lazycalc()[oidx] ^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to <lambda>(<immlib.calc object at 0x1555009b8b40>, (6, 7, 8, (0, 0), (3, 0))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 855, in _source_lookup val = lazycalc()[oidx] ^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to <lambda>(<immlib.calc object at 0x15550096a4c0>, (6, 7, 8, (1, 3), (1, 2), (6, 0), (1, 1), (1, 0), 24, 25, 26, 27, 28)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 884, in <lambda> lazy(lambda c,args: f(inputtup, calctup, c, args), c, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 873, in _call_calc r = c.eager_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 448, in eager_call res = self.function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/tasks/tractography.py", line 309, in fiber_odf params_file = data_imap[f"{odf_model.lower()}_params"] ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 381, in __getitem__ v = v() ^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to _source_lookup((lazy(<23454799358480>: ready), lazy(<23454799358192>: ready), lazy(<23454799790816>: ready), lazy(<23454799799648>: ready), lazy(<23454799800320>: ready), lazy(<23454799799552>: ready), lazy(<23454799798016>: ready), lazy(<23454799799360>: ready), lazy(<23454799799792>: ready), lazy(<23454799795904>: ready), lazy(<23454799800176>: waiting), lazy(<23454799801040>: waiting), lazy(<23454799801376>: waiting), lazy(<23454799801568>: waiting), lazy(<23454799801664>: waiting), lazy(<23454799801808>: waiting), lazy(<23454799801184>: waiting), lazy(<23454799802144>: waiting), lazy(<23454799800560>: waiting), lazy(<23454799791968>: waiting), lazy(<23454799802288>: waiting), lazy(<23454799802480>: waiting), lazy(<23454799802672>: waiting), lazy(<23454799802864>: waiting), lazy(<23454799803056>: waiting), lazy(<23454799803248>: waiting), lazy(<23454799803440>: waiting), lazy(<23454799803632>: waiting), lazy(<23454799803824>: waiting), lazy(<23454799804016>: ready), lazy(<23454799804208>: ready), lazy(<23454799804400>: ready)), (lazy(<23454799804688>: waiting), lazy(<23454799804784>: ready), lazy(<23454799804880>: waiting), lazy(<23454799804976>: ready), lazy(<23454799805072>: waiting), lazy(<23454799805168>: waiting), lazy(<23454799805264>: waiting), lazy(<23454799805360>: waiting), lazy(<23454799805456>: waiting), lazy(<23454799805552>: waiting), lazy(<23454799805648>: waiting), lazy(<23454799805744>: waiting), lazy(<23454799805840>: waiting), lazy(<23454799805936>: waiting), lazy(<23454799806032>: waiting), lazy(<23454799806128>: waiting), lazy(<23454799806224>: waiting), lazy(<23454799806320>: waiting), lazy(<23454799806416>: waiting), lazy(<23454798626928>: waiting), lazy(<23454798627024>: waiting), lazy(<23454798627120>: waiting), lazy(<23454798627216>: waiting), lazy(<23454798627312>: waiting), lazy(<23454798627408>: waiting), lazy(<23454798627504>: waiting), lazy(<23454798627600>: waiting), lazy(<23454798627696>: waiting), lazy(<23454798627792>: waiting), lazy(<23454798627888>: waiting), lazy(<23454798627984>: waiting), lazy(<23454798628080>: waiting), lazy(<23454798628176>: waiting), lazy(<23454798628272>: waiting), lazy(<23454798628368>: waiting), lazy(<23454798628464>: waiting), lazy(<23454798628560>: waiting), lazy(<23454798628656>: waiting), lazy(<23454798628752>: waiting), lazy(<23454798628848>: waiting), lazy(<23454798628944>: waiting), lazy(<23454798629040>: waiting), lazy(<23454798629136>: waiting), lazy(<23454798629232>: waiting), lazy(<23454798629328>: waiting), lazy(<23454798629424>: waiting), lazy(<23454798629520>: waiting), lazy(<23454798629616>: waiting), lazy(<23454798629712>: waiting), lazy(<23454798629808>: waiting), lazy(<23454798629904>: waiting), lazy(<23454798630000>: waiting), lazy(<23454798630096>: waiting), lazy(<23454798630192>: waiting), lazy(<23454798630288>: waiting), lazy(<23454798630384>: waiting), lazy(<23454798630480>: waiting), lazy(<23454798630576>: waiting), lazy(<23454798630672>: waiting), lazy(<23454798630768>: waiting), lazy(<23454798630864>: waiting), lazy(<23454798630960>: waiting), lazy(<23454798631056>: waiting), lazy(<23454798631152>: waiting), lazy(<23454798631248>: waiting), lazy(<23454798631344>: waiting), lazy(<23454798631440>: waiting), lazy(<23454798631536>: waiting), lazy(<23454798631632>: ready), lazy(<23454798631728>: waiting), lazy(<23454798631824>: waiting), lazy(<23454798631920>: waiting)), (21, 0)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 884, in <lambda> lazy(lambda c,args: f(inputtup, calctup, c, args), c, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 868, in _call_calc for (p,arg) in zip(c.signature.parameters.values(), argvals): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 855, in _source_lookup val = lazycalc()[oidx] ^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to <lambda>(<immlib.calc object at 0x1554ff62b7c0>, (4, 2)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 855, in _source_lookup val = lazycalc()[oidx] ^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to <lambda>(<immlib.calc object at 0x1554ff62bc40>, (0, 1, 4, 2, (7, 0), (1, 0), (5, 0), 6, 7)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 884, in <lambda> lazy(lambda c,args: f(inputtup, calctup, c, args), c, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 873, in _call_calc r = c.eager_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 448, in eager_call res = self.function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/tasks/decorators.py", line 84, in wrapper_as_file gen, meta = func( ^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/tasks/segmentation.py", line 64, in segment streamlines = tractography_imap["streamlines"] ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 381, in __getitem__ v = v() ^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to _source_lookup((lazy(<23454796711776>: ready), lazy(<23454796711824>: ready), lazy(<23454796712016>: ready), lazy(<23454796712208>: waiting), lazy(<23454796712400>: ready), lazy(<23454796712592>: waiting), lazy(<23454796712784>: waiting), lazy(<23454796712976>: waiting)), (lazy(<23454796713264>: waiting), lazy(<23454796713360>: waiting), lazy(<23454796713456>: waiting), lazy(<23454796713552>: waiting), lazy(<23454796713648>: waiting), lazy(<23454796713744>: waiting), lazy(<23454796713840>: waiting), lazy(<23454796713936>: waiting), lazy(<23454796714032>: waiting)), (8, 0)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 884, in <lambda> lazy(lambda c,args: f(inputtup, calctup, c, args), c, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 868, in _call_calc for (p,arg) in zip(c.signature.parameters.values(), argvals): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 855, in _source_lookup val = lazycalc()[oidx] ^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to <lambda>(<immlib.calc object at 0x1554ff628640>, (3, 4, 5, 0, 1, 7, 8)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__ val = part() ^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 855, in _source_lookup val = lazycalc()[oidx] ^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to <lambda>(<immlib.calc object at 0x1554ff628440>, (3, 4, (0, 0), 5)) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/storage/home/hcoda1/0/ldaumail3/scratch/afq_ampb/code/9_participant_pyAFQ_gpu_gmwmi.py", line 86, in <module> main( File "/storage/home/hcoda1/0/ldaumail3/scratch/afq_ampb/code/9_participant_pyAFQ_gpu_gmwmi.py", line 74, in main myafq.export_all(xforms = False) File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/api/participant.py", line 244, in export_all export_all_helper(self, xforms, indiv, viz) File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/api/utils.py", line 148, in export_all_helper api_afq_object.export("indiv_bundles") File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/api/participant.py", line 194, in export return self.plans_dict[section][attr_name] ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 381, in __getitem__ v = v() ^^^ File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 152, in __call__ raise init_error File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 121, in __new__ raise LazyError((fn, args, kw)) pcollections._lazy.LazyError: lazy raised error during call to _source_lookup((lazy(<23454796715376>: ready), lazy(<23454796715424>: ready), lazy(<23454796715616>: waiting), lazy(<23454796715808>: ready), lazy(<23454796716000>: ready), lazy(<23454796716192>: ready), lazy(<23454796716384>: waiting), lazy(<23454796716576>: ready), lazy(<23454796716768>: ready), lazy(<23454796716960>: waiting), lazy(<23454796717152>: waiting)), (lazy(<23454796717440>: waiting), lazy(<23454796717536>: waiting), lazy(<23454796717632>: waiting), lazy(<23454796717728>: waiting), lazy(<23454796717824>: waiting), lazy(<23454796717920>: waiting), lazy(<23454796718016>: waiting), lazy(<23454796718112>: waiting)), (5, 0))

I tried to use a Dockerfile instead and generate a Docker image before converting it into an apptainer image and ended up with the same errors. One last thing I did was to try and patch niimath so that the niimath scripts do not get edited during the process. I basically added this code to the configuration Dockerfile file:

# ===============================================================
# pyAFQ GPU Dockerfile (fixed niimath + read-only FS safe)
# ===============================================================

# Base image with CUDA 12.1 and Ubuntu 22.04
FROM nvidia/cuda:12.1.0-devel-ubuntu22.04

# Avoid interactive prompts
ENV DEBIAN_FRONTEND=noninteractive

# ---------------------------------------------------------------
# System dependencies
# ---------------------------------------------------------------
RUN apt-get update && apt-get install -y \
    apt-transport-https ca-certificates gnupg software-properties-common \
    gcc git wget curl numactl cmake && \
    rm -rf /var/lib/apt/lists/*

# ---------------------------------------------------------------
# Install Miniconda
# ---------------------------------------------------------------
RUN curl -L "https://repo.anaconda.com/miniconda/Miniconda3-py311_25.1.1-1-Linux-x86_64.sh" \
      -o /tmp/miniconda.sh && \
    bash /tmp/miniconda.sh -b -p /opt/anaconda && \
    rm /tmp/miniconda.sh

# Add Conda to PATH
ENV PATH=/opt/anaconda/bin:$PATH

# ---------------------------------------------------------------
# Create and populate AFQ environment
# ---------------------------------------------------------------
RUN /opt/anaconda/bin/conda create -y -n afq python=3.11 && \
    /opt/anaconda/envs/afq/bin/pip install \
        numpy>=2.0.0 scipy>=1.13.0 cython nibabel dipy tqdm fslpy && \
    /opt/anaconda/envs/afq/bin/pip install git+https://github.com/dipy/GPUStreamlines.git && \
    /opt/anaconda/envs/afq/bin/pip install git+https://github.com/36000/pyAFQ.git@dam

# -------------------------------------------------------------------
# Patch niimath: remove chmod + indentation issues (Apptainer-safe)
# -------------------------------------------------------------------
RUN echo "🔧 Repairing niimath for Apptainer..." && \
    FILE=/opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/__init__.py && \
    # 1. Use awk to remove the entire block related to creating an executable,
    # and also remove any os.chmod calls that might be left.
    # This is less likely to break indentation than complex block deletion.
    awk ' \
      !/if os.path.isfile\(bin\):/ && \
      !/os.chmod/ && \
      !/try:/ && \
      !/bin = os.path.join/ && \
      !/with open\(bin, "w"\) as fd:/ && \
      !/fd.write\(shbang\)/ && \
      !/fd.write\(f"\s*from niimath import main\n"\)/ && \
      !/fd.write\(f"\s*if __name__ == \x27__main__\x27:\n"\)/ && \
      !/fd.write\(f"\s*main\(\)\n"\)/ \
      {print} \
    ' $FILE > ${FILE}.fixed && mv ${FILE}.fixed $FILE && \
    \
    # 2. Create a clean launcher (as before)
    echo '#!/bin/bash' > /usr/local/bin/niimath && \
    echo 'exec /opt/anaconda/envs/afq/bin/python -m niimath "$@"' >> /usr/local/bin/niimath && \
    chmod +x /usr/local/bin/niimath && \
    echo "✅ niimath patched successfully" && \
    /usr/local/bin/niimath --help || true
    
# ---------------------------------------------------------------
# Environment setup
# ---------------------------------------------------------------
ENV CONDA_DEFAULT_ENV=afq
ENV PATH=/opt/anaconda/envs/afq/bin:$PATH
ENV LD_LIBRARY_PATH=/opt/anaconda/lib:$LD_LIBRARY_PATH

# ---------------------------------------------------------------
# Default command (runs Python in AFQ env)
# ---------------------------------------------------------------
ENTRYPOINT ["/opt/anaconda/envs/afq/bin/python"]

Though I still end up with an indentation error:

INFO:AFQ:No stop mask given, using ACT and 3T prob maps esimated from T1w
INFO:AFQ:Calculating _desc-bundles_tractography
INFO:AFQ:Calculating _desc-T1w_mask.nii.gz
ERROR:AFQ:Command failed: ['niimath', '/tmp/tmpkqejk7v_/t1.nii.gz', '-conform', '-gz', '0', '-', '-odt', 'char']
ERROR:AFQ:Return code: 1
ERROR:AFQ:STDERR:
Traceback (most recent call last):
  File "/opt/anaconda/envs/afq/bin/niimath", line 3, in <module>
    from niimath import main
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/__init__.py", line 5
    from ._dist_ver import __version__
IndentationError: unexpected indent

Traceback (most recent call last):
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/pcollections/_lazy.py", line 143, in __call__
    val = part()
          ^^^^^^
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 884, in <lambda>
    lazy(lambda c,args: f(inputtup, calctup, c, args), c, args)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 873, in _call_calc
    r = c.eager_call(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/immlib/workflow/_core.py", line 448, in eager_call
    res = self.function(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/tasks/decorators.py", line 84, in wrapper_as_file
    gen, meta = func(
                ^^^^^
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/tasks/data.py", line 1426, in t1_brain_mask
    return run_brainchop(nib.load(t1_file), "mindgrab"), dict(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/nn/brainchop.py", line 68, in run_brainchop
    volume, header = _run_brainchop_command(conform, [tmp_t1_file])
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/AFQ/nn/brainchop.py", line 36, in _run_brainchop_command
    return func(*args)
           ^^^^^^^^^^^
  File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/brainchop/niimath/__init__.py", line 273, in conform
    res = subprocess.run(cmd, capture_output=True, check=True)

... etc

Any idea what is causing so much trouble with niimath?

Thanks for your time and help!
Best,
-Loïc

[arokem]

Hi @ldaumail : thanks for posting this detailed report. Just for clarity, this is using the code in #103, correct? So this is not an issue for users of the main branch right now, but would be an issue if #103 is merged.

I wonder whether you want to separately go through the compilation/installation process for niimath, as detailed here: https://github.com/rordenlab/niimath/?tab=readme-ov-file#compilation, in the context of building your Docker/Apptainer container.

Also, adding @neurolabusc : if you have a moment, your input would be appreciated here. The TLDR is that we are trying to use brainchop in our software, but niimath permissions are a bit wonky in Docker/Apptainer. I wonder if you've seen something like this before, or if you have any thoughts on cause/solution. Thanks!

[ldaumail]

Hi @arokem, thanks for your prompt reply. my bad, yes this is using the code in #103. I would like to obtain wmgmi streamline endpoints from the tractography outcome. With pip install git+https://github.com/36000/pyAFQ.git@dam

I will look into niimath, thanks for the suggestion!

[neurolabusc]

@arokem this is outside my expertise. niimath is a minimal executable that is natively compiled for each environment. Some installations (e.g. pip install) create a shim for calling it, which can add an extra layer of complexity. You may want to consult a container expert to understand the specific issue here. I believe your use case is due to the fact that brainchop includes niimath as a dependency, in which case the enyclopedic knowledge of @sergeyplis might be helpful.

[sergeyplis]

@arokem are you using the latest version of brainchop from pypi? I had a problem fixed recently neuroneural/brainchop-cli#34 that could have caused this. If still does not work, would it be possible to share the /tmp/tmp5m0g1i1o/t1.nii.gz input?

[spikedoanz]

line 3, in from niimath import main File "/opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/init.py", line 38, in os.chmod(bin, os.stat(bin).st_mode | 0o111) OSError: [Errno 30] Read-only file system: '/opt/anaconda/envs/afq/lib/python3.11/site-packages/niimath/bin/niimath'

niimath seems to be failing because the niimath python wrapper is attempting to modify the file permissions inside the anaconda container at runtime, instead of at install time. in a docker environment, this definitely might break especially if the anaconda environment is mounted ONTO the container instead of installed WITHIN the container.

https://github.com/rordenlab/niimath/blob/master/niimath/__init__.py#L36-L38

the correct way of handling this is to:

1. remove the runtime file permission change in niimath/init.py
2. instead have the file permission change wrapped into the setup steps

for an intermediate fix, i believe installing brainchop WITHIN the docker container should make the OS permission issues go away.

@ldaumail

[arokem]

@spikedoanz : I might be missing something, but I believe that this line in the Dockerfile:

    /opt/anaconda/envs/afq/bin/pip install git+https://github.com/36000/pyAFQ.git@dam

is trying to install niimath into the container, because that branch of the software requires brainchop: https://github.com/tractometry/pyAFQ/pull/103/files#diff-fa602a8a75dc9dcc92261bac5f533c2a85e34fcceaff63b3a3a81d9acde2fc52

[spikedoanz]

hmm. i'm not sure what the semantics of invoking anaconda pip install inside of a container. could you try using uv or a raw python -m venv installation instead?

[spikedoanz]

pip install git+https://github.com/spikedoanz/niimath.git@niimath-permissions

could you overriding the niimath install from this origin? i've implemented the patch i suggested above there.

[arokem]

Worth trying!

@ldaumail : I think you'd want to try adding a:

/opt/anaconda/envs/afq/bin/pip install -U git+https://github.com/spikedoanz/niimath.git@niimath-permissions 

right after the pyAFQ installation line.

Thanks!

[ldaumail]

Thanks for all your help!

@spikedoanz and @arokem

I tried just now, but it doesn't seem to work.

+ /opt/anaconda/envs/afq/bin/pip install -U git+https://github.com/spikedoanz/niimath.git@niimath-permissions
Collecting git+https://github.com/spikedoanz/niimath.git@niimath-permissions
  Cloning https://github.com/spikedoanz/niimath.git (to revision niimath-permissions) to /tmp/pip-req-build-61xwiqk0
  Running command git clone --filter=blob:none --quiet https://github.com/spikedoanz/niimath.git /tmp/pip-req-build-61xwiqk0
  Running command git checkout -b niimath-permissions --track origin/niimath-permissions
  Switched to a new branch 'niimath-permissions'
  Branch 'niimath-permissions' set up to track remote branch 'niimath-permissions' from 'origin'.
  Resolved https://github.com/spikedoanz/niimath.git to commit e7d9a166c42d3b201a922c8f3fcc0b0d255c62ad
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: niimath
  Building wheel for niimath (pyproject.toml) ... done
  Created wheel for niimath: filename=niimath-0.1.dev264+ge7d9a166c-cp311-cp311-linux_x86_64.whl size=493514 sha256=0a661f57ffca8d4571af8ee94f621f4add3c1eec46949cd3b1a3a3d93ddb8744
  Stored in directory: /tmp/pip-ephem-wheel-cache-zsqdjoci/wheels/81/39/31/6da5919949681ee7307fc50a63c9e41075fa3028f7e5e4b48d
Successfully built niimath
Installing collected packages: niimath
  Attempting uninstall: niimath
    Found existing installation: niimath 1.0.20250804
    Uninstalling niimath-1.0.20250804:
      Successfully uninstalled niimath-1.0.20250804
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
brainchop 0.1.23 requires niimath>=1.0.20250804, but you have niimath 0.1.dev264+ge7d9a166c which is incompatible.
Successfully installed niimath-0.1.dev264+ge7d9a166c

I guess I also need to change something in the niimath/init.py script?

[ldaumail]

Quick update: the jobs seem to run now (for longer than it was before, still hasn't crashed). Even with the error I had above.

Will keep you updated. Thanks a lot!

[spikedoanz]

yea i would invalidate the cache of the docker container just in case. but i'm glad to hear it's working!

[ldaumail]

@spikedoanz quick follow up question, how would you "invalidate the cache of the docker container "?

by doing something like:
docker build --no-cache -t my_image:latest .

or like:
ARG CACHE_BREAKER=1
before the niimath layer I want to force rebuild?

Best,
-Loïc

[spikedoanz]

both of those options should work. i would also wipe any installs of niimath, or the conda environments created related to the project just in case it tries to read from the old copies.