MultiFile plugin

The MultiFile plugin allows you import a ZIP archive of field data files intended for other plugins already enabled on your AQTS app server.

The main benefit of the MultiFile plugin is that it can intercept same-day/same-location visit activities, combine them into a single visit, and import the combined result into AQTS.

Note: If you want to use the TabularCsv plugin from inside the MultiFile plugin, you'll need to use the MultiFile.Configurator.exe tool in order to expose your TabularCsv TOML configurations through the MultiFile parsing sequence.

How the MultiFile plugin works with "delegate" plugins

When the MultiFile plugin is installed and enabled, it first attempts to discover any other plugins also installed on the app server. It "discovers" other plugins simply by looking at the immediately adjacent folders of the app server file system for any .NET DLLs which are also field data plugins.

Any other plugins discovered this way are called "delegate" plugins, and the MultiFile will ask each delegate plugin to try to parse each file in the ZIP archive. When a delegate plugin attempts to parse a file, the delegate plugin thinks it is just part of a normal AQTS field data pipeline. Delegate plugins don't really know that they aren't talking to real AQTS server.

The basic processing pipeline for the MultiFile plugin is:

• Exit with CannotParse if the file to be parsed is not a ZIP archive.
• Exit with CannotParse if the ZIP archive contains any nested folders or files nested in folders.
• Exit with CannotParse if the ZIP archive does not contain any files at the root of the archive.
• Discover the delegate plugins, either from its own "Config" setting, or by inspecting adjacent folders if the Config setting is blank/empty.
• Exit with CannotParse if no delegate plugins are discovered.
• Order the delegate plugins by PluginPriority, from lowest value to highest value.
• For each file in the ZIP archive:
  • For each delegate plugin:
    • Try to parse the file using the delegate plugin.
    • If the delegate plugin returns returns SuccessfullyParsedAndDataValid, then save the created visit in memory.
  • Exit with CannotParse if no delegate plugin can parse the file.
• Exit with CannotParse if no visits have been parsed.
• Combine any same-day/same-location in-memory visits into a single visit for that day, adjusting the visit start/end times to contain all the activities.
• Send all the combined visits into the plugin framework.
• Exit with SuccessfullyParsedAndDataValid.

Configuring the MultiFile plugin

Many field data plugins either require configuration, or can use optional configuration settings if their default behaviour doesn't work for your organization.

Configurable plugins include:

• TabularCsv - This plugin has no default settings, so it absolutely must be configured in order to do useful work.
• Teledyne SxS Pro has default behaviour which can be configured.
• FlowTracker2 has default behaviour which can be configured.
• USGS QRev has default behaviour which can be configured.
• OTT QReview has default behaviour which can be configured.
• AquaCalc 5000 has default behaviour which can be configured.
• Water Survey Canada eHSN has default behaviour which can be configured.

If you use an plugin which requires configuration settings AND you need to use the MultiFile plugin to combine data from separate field visit files, then you will need to use the MultiFile.Configurator.exe tool to expose the delegate plugin settings through the MultiFile context.

Why MultiFile?

The MultiFile plugin exists because field visit data can often arrive in multiple file formats (eg. A FlowTracker2 measurement file, plus some custom field visit CSV from your organization). If both bits of field data occured on Wednesday at the same location, ideally you want both activities represented as a single AQTS field visit.

This scenario (the same day, same location, but different source data files) is what we call the visit merge problem, and it isn't quite clear how to solve it for all possible customers, so AQTS doesn't allow it out of the box.

In an ideal world, the MultiFile plugin should not be necessary. In fact, you should check your AQTS release notes to see if the visit merge problem has finally been solved. If it has, just delete the MultiFile plugin and use the built-in functionality, since that will offer the most robust & supportable workflow for your organization.

What if I don't use the MultiFile plugin?

If you don't use the MultiFile plugin to import activities from multiple files which occured on the same day at the same location, AQTS will still work, but your field visit activities will be contained by multiple visits on that day.

This isn't the end of the world by any means. Your discharge measurements from multiple same-day visits will still show in Rating Review Tool and any field visit readings will still be visible in the Data Review Tool.

The only quirk that results is your field visit approval workflow will require a bit more due diligence. If your technicians are used to answering the question "Is the Wednesday visit approved?" then they will need to see if more than one visit occurs on Wednesday. That isn't too hard to do, but it is subtle and error prone.

So the MultiFile plugin was created by the Aquatic Informatics Applications Engineering team to reduce the chances that two Wednesday visits are created.

Can I use MultiFile to merge new data with existing FV data?

Nope.

MultiFile can only be used when all the FV data you want to import doesn't yet exist in AQTS.

• Field data plugins do not have access to any other field data or most other parts of the system.
• Field data plugins can only read the input file content and decide how to transform that content into a new set of AQTS field visits.

Once the core AQTS product solves the the visit merge problem, then the MultiFile plugin can be disabled.

As of AQTS 2022.2, the MultiFile plugin is still required for some organization workflows.