Parallel processing on Sender Module.

[armghan]

OpenAs2Server v3.11.0
I want to create outbound files to process in parallel, rather than in series.

I have added the following configuration to config.xml:

<module classname="org.openas2.processor.sender.AS2SenderModule" thread_pool_size="500" max_connections="1000"/>

I’ve also updated the Directory Polling Module with the following settings:

<module classname="org.openas2.processor.receiver.AS2DirectoryPollingModule" interval="1" outboxdir="$properties.storageBaseDir$/Outbound/PartnerB" errordir="$properties.storageBaseDir$/Outbound/PartnerB/errors" defaults="sender.as2_id=PartnerA-AS2-T, receiver.as2_id=PartnerB-AS2-T" sendfilename="true" mimetype="application/EDI-X12" process_files_in_parallel="true" max_parallel_files="20" poll.parallel="true" poll.threads="50" />

I have enabled DEBUG logs to verify if the outbound processing is occurring in parallel, but it still appears to be working in series.

In the logs, I consistently see the following line for each file, indicating sequential processing:

FINER org.apache.http.headers: http-outgoing-0 >>

Can someone confirm if I’m missing something or if OpenAS2 version 3.11 does not support parallel sending in the Sender module?

[uhurusurfa]

Parallel processing was added in 3.7.0 so your version should support parallel processing:
https://github.com/OpenAS2/OpenAs2App/blob/master/changes.txt#L111

Note that the parallel processing simply launches the exact same file processing method as for serial processing - just using a thread:

                    // if the file length has stayed the same, process the file
                    // and then stop tracking it
                    if (processFilesAsThreads) {
                        executorService.execute(new Runnable() {
                            @Override
                            public void run() {
                                processSingleFile(file, fileEntryKey);
                            }
                        });

                    } else {
                        processSingleFile(file, fileEntryKey);
                    }

So you will not see anything different in the logs. I will publish as new release shortly that will provide a DEBUG log mesage indicating whether it is in parllel mode or not.

I strongly recommend you upgrade to the latest version and switch to using a separate properties file for configuring the core aspects of OpenAS2 so that future upgrtades are simple. See the "OpenAS2 Configuration Properties" section (section 7.4.1 in the latest OpenAS2HowTo.pdf) for configuring your system using properties.

[armghan]

Hi @uhurusurfa, thank you for your response to my earlier query.

I'm in the process of upgrading OpenAS2Server from version 2.9.0 to 3.11.0, which was integrated into our product a few years ago. I expected performance improvements—specifically in outbound file processing speed—by using the new parallel polling parameters in the DirectoryPollingModule section of config.xml.

However, I haven’t observed any noticeable increase in outbound processing speed between versions 2.9 and 3.11.

Could you please help confirm if I’m configuring this correctly or if there’s something I might be missing?

I'll soon be upgrading to the latest version as I'm also evaluating 4.5.2 for the integration in our product.

[uhurusurfa]

The parallel processing improves things when you have multiple simultaneous files to process - it will have no effect on the overall lifecycle time of picking up the file and sending it to your partner.

This is because there is a long time spent waiting for the partner to process the file and return an MDN when in SYNC MDN mode which is the most common. It is during the time spent waiting for a response that other files also get processed and sent making it more efficient in terms of use of your servers processing capability.

The only thing you can do to speed up sending a file is to speed up the time taken to encrypt and sign the AS2 message containing your file by allocating more RAM and it will only work if your RAM is already over utilized due to the size of your files you are sending relative to the amount of RAM allocated to the Java VM.

You would have to analyse exactly where things appear to be going slowly and then you can figure out what options you have to speed it up.