NF does not perform proxy authentification and cannot access github

[kpetrov]

Bug report

Doesn't authenticate at the proxy server, no internet access

Expected behavior and actual behavior

We have a computing cluster with internet access provided via proxy. Most programs take the credentials from the environmental variables and work just fine. Nextflow doesn't try to authenticate.

Steps to reproduce the problem

install nextflow, install graalvm, run
)->nextflow run nf-core/demo -profile maestro --input samplesheet.csv

Program output

N E X T F L O W ~ version 24.10.3

Pulling nf-core/demo ...
WARN: Cannot read project manifest -- Cause: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
Cannot find nf-core/demo -- Make sure exists a GitHub repository at this address https://github.com/nf-core/demo

Environment

• Nextflow version: 24.10.3
• Java version: graalvm 22.3.1
• Operating system: Linux RHEL 8.6
• Bash version: 4.4.20

http_proxy=http://user:mysecretkeyhere@squid:3128

Additional context

Proxy is required for security and audit.

[bentsherman]

Can you share your Nextflow log? You should see these two lines if http_proxy is set:

Setting http proxy: ProxyConfig[...]
Setting http proxy authenticator

[kpetrov]

nextflow.log

thanks for a quick reply, here is the log with some debug info

[bentsherman]

From the first lines of that log it looks like it is setting the proxy credentials for http and https. So I'm not sure why it would fail

[kpetrov]

can I provide any further information? if so, which?

[menzel]

I have the same issue, password/user and proxy server url are shown fine the log, when Nextflow tries to use it, it fails with "Authentication required".

Is there a workaround the download the file manually and place it somewhere for Nextflow to find?

[bentsherman]

If it's the git repo for the pipeline, you can just clone the repo via git or some other way that works, then run the pipeline locally

[braffes]

After a lot of compilation, I finally found the issue.

Firstly, I had to add the following line to the launch.sh script:

JVM_ARGS+=" -Djdk.http.auth.tunneling.disabledSchemes=''"

This is due to a behavior change in Java 8u111, as described in the release notes:
"Disable Basic authentication for HTTPS tunneling

In some environments, certain authentication schemes may be undesirable when proxying HTTPS. Accordingly, the Basic authentication scheme has been deactivated, by default, in the Oracle Java Runtime, by adding Basic to the jdk.http.auth.tunneling.disabledSchemes networking property. Now, proxies requiring Basic authentication when setting up a tunnel for HTTPS will no longer succeed by default. If required, this authentication scheme can be reactivated by removing Basic from the jdk.http.auth.tunneling.disabledSchemes networking property, or by setting a system property of the same name to an empty string ("")) on the command line.

Regarding the code, there is also an issue here:

nextflow/modules/nextflow/src/main/groovy/nextflow/util/ProxyConfig.groovy

Lines 62 to 63 in ac71280

	if( protocol && !protocol.equalsIgnoreCase(getRequestingProtocol()) )
	return null

For some reason, getRequestingProtocol() always returns "http" when the protocol is actually "https". To test this, I added the following lines inside getPasswordAuthentication():

System.out.println("protocol");
System.out.println(protocol);
System.out.println("getRequestingProtocol()");
System.out.println(getRequestingProtocol());
System.out.println("getRequestingURL()");
System.out.println(getRequestingURL());

This produced the following output:

[braffest@maestro-submit nextflow]$ NXF_PLUGINS_MODE=prod ./launch.sh  run https://github.com/nf-core/demo.git -profile maestro  --input samplesheet.csv  --outdir out      
                                                                                                                                                                                        
 N E X T F L O W   ~  version 24.10.5                                                                                                                                                   
                                                                                                                                                                                        
protocol                                                                                                                                                            
https
getRequestingProtocol()
http
getRequestingURL()
https://github.com/nf-core/demo.git/info/refs?service=git-upload-pack
Launching `https://github.com/nf-core/demo` [backstabbing_poitras] DSL2 - revision: 04060b4644 [master]    

Looking at the Nextflow code, I see that the connection is an instance of HttpURLConnection. The Java code suggests that getRequestingProtocol() may not be able to return "https".

To fix this issue, I replaced the line:

nextflow/modules/nextflow/src/main/groovy/nextflow/util/ProxyConfig.groovy

Lines 62 to 63 in ac71280

	if( protocol && !protocol.equalsIgnoreCase(getRequestingProtocol()) )
	return null

with:

if ((protocol && !protocol.equalsIgnoreCase(getRequestingProtocol())) && !(protocol == 'https' && getRequestingProtocol() == 'http')) {
    return null
}

This change fixed the issue.

I probably can do a PR, but I am not sure about if something should be done on jdk.http.auth.tunneling.disabledSchemes directly in the code.

[marc-sturm]

We have the same issue behind our authentificating proxy.
If there is a general solution, I'm happy to test it.