Skip to content

MultiFile.Configurator.exe

Jump to bottom
Doug Schmidt edited this page Aug 4, 2022 · 11 revisions

The MultiFile.Configurator.exe tool helps you quickly configure the MultiFile plugin so that each delegate plugin is able to access your organization's custom settings.

  • The tool creates a duplicate setting for each delegate plugin that the MultiFile plugin is able to "see". The MultiFile plugin then uses these settings when it launches each delegate plugin to parse candidate files correctly.
  • The tool should be run whenever the settings for a delegate plugin are changed.
  • The generated config setting can be manually edited for finer control over the behaviour of the MultiFile plugin or any of the delegate plugins it uses.
SettingGroup SettingKey SettingValue Description
FieldDataPluginConfig-MultiFile Config The JSON document described in the JSON configuration section. All the settings for delegate plugins are stored within this one setting.

Yep, this all feels very brittle. But that's why the Applications Engineering group made this tool, to try to automate the effort of keeping these settings in sync. Our hope is that most customers who need to use the MultiFile plugin can use the MultiFile.Configurator.exe tool to automatically configure the MultiFile plugin to match their other configurations.

See the example use cases for this might be used in practice.

JSON Configuration

{
    "Plugins": [
        {
            "Path": "path to installed folder or *.plugin file",
            "PluginPriority": 0,
            "Settings": {
                "Key": "value"
            }
        }
    ],
    "Verbose": false
}

The Plugins[i].Path property

The Path property can be:

  • blank/missing (which causes the Plugin[i].Settings property to be treated slighty differently)
  • the filename of a *.plugin file, with no directory component. (eg. "TabularCsv.plugin" or "SxSPro.plugin") This is the preferred approach for running the within the FieldVisitHotFolderService context.
  • an absolute path to the extracted plugin folder on the app server (eg. "C:\ProgramData\Aquatic Informatics\AQUARIUS Server\FieldDataPlugins\343af51d4e2ef249bc2f23b2b072e02f4254a1d0cdaed03c031c64de19132a28")

The Plugins[i].PluginPriority property

The PluginPriority property is optional, and will only be used when the Path property is not blank/empty.

If omitted, the delegate plugin will default to int.MaxValue, causing the plugin to run las, after all other delegate plugins have a chance to parse a file.

  • The first delegate plugin to successfully parse a file "wins" and no other delegate plugins will be run.
  • Delegate plugins will be run from lowest PluginPriority value to the highest value.
  • Delegate plugins at the same priority value will be run at essentially a random order. That is usually fine.
  • PluginPriority values do not need to be consecutive. It is totally fine to have gaps, with say the TabularCsv plugin running at priority 10 and the QRev plugin running next at priority 200.

The Plugins[i].Settings property

The Settings dictionary has key/value pairs which are fed to each of the delegate plugins.

When a plugin is running directly with AQTS, and not as a delegate, the settings for a plugin are pulled from the group whose name matches the FieldDataPlugin-{PluginFolderName} group. All the key/value pairs within the group are made available to the plugin.

The format of the key depends on whether the Plugin[i].Path is blank/empty or not.

  • When the Path is blank/empty, the Setting keys use the {DelegatePluginFolderName}-{Key} pattern. The MultiFile plugin will strip out the {DelegatePluginFolderName}- prefix from the key, so that the delegate plugin will only see the {Key} as usual.
  • When the Path is not blank/empty, then the delegate plugin is already known, so the Setting key/value pairs are used as-is.

Set Verbose to true to trace through the plugin logic

The Verbose boolean setting defaults to false. When true, the MultiFile plugin will log a bit more information about the delegate plugins it discovers and their location. The effect of the MultiFile plugin isn't changed by the value of the Verbose setting, but it can be useful to trace through the logic when files are not parsing as expected.

When MultiFile is run on the AQTS app server, the extra log lines will end up in FieldDataPluginFramework.log. When MultiFile is run from the FieldVisitHotFolderService, then extra log lines will end up in FieldVisitHotFolderService.log.

Example use cases

The two main use cases are:

Both examples will try to configure a that uses the MultiFile plugin to import ZIPs of:

  • FlowTracker2 measurements (*.ft files) with a 95% confidence interval ISO uncertainty, instead of the deafult 68% confidence interval.
  • a CSV of other parameter readings, using a TabularCsv configuration.

The custom CSV for readings looks like this:

Location, Time, Value, ParameterId, UnitId
LOC1, 2020-Jan-12 12:35, 20.5, TA, degC
LOC2, 1988-Aug-8 15:10, -3.5, TA, degC

And TabularCsv is configured to parse that file useing this 7-line TOML configuration:

Location = '@Location'
Time = '@Time'

[Reading]
Value = '@Value'
ParameterId = '@ParameterId'
UnitId = '@UnitId'

On the AQTS app server, there are two settings for these plugins:

SettingGroup SettingKey SettingValue
FieldDataPluginConfig-FlowTracker2 IsoUncertaintyScalar 1.96
FieldDataPluginConfig-TabularCsv SimpleReadings Location = '@Location'\nTime = ...

Use case 1 - Configuring MultiFile to run on the app server

This is main expected use case, where a customer creates a ZIP of two or more field visit files with data for the same day and location.

The customer is using MultiFile to ensure that all the same-day-same-location data from all the files just create a single visit, instead of one visit per file, and the ZIP files are being uploaded directly to Springboard.

Just launch the tool with credentials for your app server, and the tool will create a setting with Group=FieldDataPluginConfig-MultiFile Key=Config and a Value property generated from all the enabled and configured plugins on the app server.

MultiFile.Configurator.exe -server=doug-vm2019

11:58:07.532 INFO  - Connecting to doug-vm2019 ...
11:58:16.161 INFO  - Connected to doug-vm2019 (2022.1.85.0) as admin
11:58:16.306 INFO  - Saving generated configuration to the app server ...
11:58:16.308 INFO  - Creating new MultiFile Config setting ...
11:58:16.471 INFO  - Saved generated configuration to the app server.

The generated JSON will just have duplicate settings for all the configured plugins on the app server. The JSON won't have Plugins[i].Path or Plugins[i].PluginPriority properties. It will only contain Plugin[i].Settings collections.

The generated JSON configuration for use case 1 looks like this:

{
    "Plugins": [
        {
            "Settings": {
                "TabularCsv-SimpleReadings": "Location = '@Location'\nTime = '@Time'\n\n[Reading]\nValue = '@Value'\nParameterId = '@ParameterId'\nUnitId = '@UnitId'",
            }
        },
        {
            "Settings": {
                "FlowTracker2-IsoUncertaintyScalar": "1.96"
            }
        }
    ],
    "Verbose": false
}

Use case 2 - Configuring MultiFile to run externally

This use case is typical for the FieldVisitHotFolderService, where the FVHFS is running some plugins locally, outside of the app server.

The -GenerateForExternalUse=true and -JsonPath=somePath options are used to generate a JSON file that can be fed into the FVHFS for its local copy of MultiFile. The JSON will contain the plugin priorities and settings from all enabled plugins on the AQTS app server.

MultiFile.Configurator.exe -server=doug-vm2019 -GenerateForExternalUse=true -JsonPath=MySettings.json

14:42:31.813 WARN  - Disabling /SaveOnServer=true since /GenerateForExternalUse=true
14:42:31.825 INFO  - Connecting to doug-vm2019 ...
14:42:32.423 INFO  - Connected to doug-vm2019 (2022.1.85.0) as admin
14:42:32.502 INFO  - Saved generated configuration to 'MySettings.json'.

The generated JSON will just have duplicate settings, priorities, and expected plugin paths for all the configured plugins on the app server.

The generated JSON configuration for use case 2 looks like this:

{
    "Plugins": [
        {
            "Path": "TabularCsv.plugin",
            "PluginPriority": 2900,
            "Settings": {
                "SimpleReadings": "Location = '@Location'\nTime = '@Time'\n\n[Reading]\nValue = '@Value'\nParameterId = '@ParameterId'\nUnitId = '@UnitId'"
            }
        },
        {
            "Path": "FlowTracker2.plugin",
            "PluginPriority": 2910,
            "Settings": {
                "IsoUncertaintyScalar": "1.96"
            }
        }
    ],
    "Verbose": false
}

Configure the FVHFS to use this JSON file via the /PluginSettings=MultiFile=Config=@somepath\MySettings.json option to allow FVHFS to supply that one setting to the MultiFile plugin, which will in turn use those settings for the appropriate delegate plugins also running within the FVHFS.

Clone this wiki locally