InfluxDB publisher: Questions about HardwareID assumptions

[ottojo]

Hi! I would like to use the InfluxDB output added in #425, and am stumbling over some assumptions made about the hardware ID. It looks like statusToInfluxLineProtocol and splitHardwareID expect the hardware ID in a form of <namespace>/<node name>, <node name> or <ignored>/<namespace>/<remaining parts as node name>. The diagnostic updater however does not set the hardware ID, but puts the node name in the diagnosticstatus name field <node name>: <diagnostic name> (https://github.com/ros/diagnostics/blob/ros2/diagnostic_updater/src/diagnostic_updater.cpp#L183).

REP 107 standardizes the latter naming scheme, and only mentions "things like serial numbers" for the hardware ID.

Is there a specific reason for using the hardware id instead of the component name as measurement name in InfluxDB? (Or an example of how component names / hardware IDs are assigned for use with the influxdb output?

[ct2034]

@dwffls Could you please comment on this?
Would it be feasible to make this configurable?

[dwffls]

@ottojo Thats a good question, the main reason is that for the usecase we had we needed the namespace for our grafana dashboards. At the time i didn't know about the use_fqn parameter to implement it in the name, so we misused the hardwareID to set it to the fully qualified name of the node.

Looking back at it you are absolutely right, we should split up the name instead of the hardwareID.
My suggestion is for the following diagnostics msg:

level: "0"
name: '/ns1/ns2/node_name: diagnostic_name'
message: Status Message
hardware_id: '123456'
values:
  - key: key1
    value: 'value1'
  - key: key2
    value: '42'

We convert it to an influx line like:

node_name,ns=ns1/ns2,name=diagnostic_name,hardware_id=123456 level=0,message="Status Message",key1="value1",key2=42 1672531200123456789

This would keep all the measurement values the same but add additional data to the influxDB.
If I have some time free this week to start on this implementation, I'll keep you posted.

@ct2034 In the documentation I couldn't find any mention of the use_fqn parameter, is this something that could be added? I'm guessing this should be done in the README of the diagnostics_updater.

[dwffls]

So after trying the fix above it appears that the diagnostic aggregator also changes the name of the status in the diagnostics array, based on the settings used in the aggregator.

For example:

- level: "\0"
  name: '/arm1/arm_communicator: errors'
  message: No errors
  hardware_id: /arm1/arm_communicator
  values:
  - key: Board Temp
    value: '33.2754'
  - key: Motors in error
    value: '0'

Is changed to:

- level: "\0"
  name: '/Arm/Hardware/Control/ arm1 arm_communicator: errors'
  message: No errors
  hardware_id: /arm1/arm_communicator
  values:
  - key: Board Temp
    value: '32.6309'
  - key: Motors in error
    value: '0'

We could fix this by listening to the the original /diagnostics message, however this also leads to a second subscriber on all diagnostics messages. And this would probably require a aggregator of its own as to reduce network traffic.

Not sure what the best way forward is for this

[dwffls]

After thinking about it for a second, I suggest we drop the send.agg option and focus on sending every msg received on the /diagnostics topic. We could build a string from every received message and send it on a timer to function as an aggregator.

This would also remove the possibility for the same datapoint to be sent with multiple analyzers, thus reducing duplicate datapoints in influxdb. Next to this it also creates the possiblity for users to send diagnostics on a higher data rate then the standard of the diagnostic_aggregator.

See pull request #525 for the implementation