AM32 Mavlink Settings Protocol

[dakejahl]

Sorry for tagging so many but I value each of your opinions and want to make sure we get as many eyes on this as possible. This is a follow up from mavlink/mavlink#2324 where you can read more about DShot Programming Mode.

@hamishwillee
@tridge
@peterbarker
@andyp1per
@dagar
@julianoes
@MaEtUgR
@sfuhrer
@DonLakeFlyer
@AlkaMotors

[DonLakeFlyer]

I'm not sold on the reasoning behind not having these come across the parameter protocol. This is what PX4 does with canbus params which makes things super easy to work with. I wouldn't create individual components for each ESC. I would create a single component id for escs. And then use FOO#_BAR naming for parameters just like many other params do. Battery params for example. Not sure why this would affect flash size? I wouldn't make these real params on the fw side. I would just layer the parameter protocol over the AM32 protocol.

[dakejahl]

Not sure why this would affect flash size? I wouldn't make these real params on the fw side. I would just layer the parameter protocol over the AM32 protocol.

Good point, you're right we don't need to make these params on the FC side. That also removes the can of worms issue of the FC "owning" the AM32 parameters.

My main concern about using parameters it that the existing mechanisms for writing parameters follow the no-save model -- as soon as a slider/checkbox is pressed the parameter is sent to the flight controller. The flight controller would ACK this parameter indicating that the value was set. This is at conflict with the save model that is used to actually write the ESC settings. Whether we're using "dshot programming mode" or "four way" both protocols need to start, write, save, stop. There are some options to work around this:

• Perform the full programming sequence per-setting. If a single setting is changed in QGC the emission of the single parameter would trigger the flight controller to enter programming mode, write the setting, save, and exit programming mode. This has some overhead, the save command causes the ESC to become unresponsive for at least 100ms. I'm not sure about the overhead with four-way mode but it might be even more since the ESC needs to reboot into the bootloader for four-way.

• Batch the parameter changes on the FC and only send them to the ESC driver once the user has clicked "write". This is at conflict with the no-save model, the FC will ACK when it receives the param value but it does not indicate that it's actually been written. There would be no way to differentiate between written and unwritten values. We would still need a MAV_CMD to trigger the write.

• Update QGC parameters model to allow using a save model. I know in the past there has been an attempt to extend the parameter protocol to support the concept of parameter groups which should be saved atomically (this was implemented in AMC for IP radio settings for example)

Or maybe I'm not thinking this through completely? Can you help me work through this? I am definitely in favor of re-using things that already exist, but I don't want to shoehorn it.

[DonLakeFlyer]

You could state that ESC parameters use the MAV_CMD_PREFLIGHT_STORAGE.param1 = PARAM_WRITE_PERSISTENT mavlink command to the ESC component to do the actual write to the ESCs. Prior to that the values are kept in temp in-memory storage on the firmware. You could even use PARAM_READ_PERSISTENT to reset the values back to the original ESC stored values to cancel in progress changes.

[dakejahl]

You could state that ESC parameters use the MAV_CMD_PREFLIGHT_STORAGE.param1 = PARAM_WRITE_PERSISTENT mavlink command to the ESC component to do the actual write to the ESCs

I wasn't aware of this command, that should work. We'd need to add MAV_COMP_ID_ESC to the MAV_COMPONENT enum.

[DonLakeFlyer]

The only issue I see with the pre-flight storage thing is that GCS will need to decide how to handle esc parameters in their main parameter editor ui:

• Don't show ESC params there, since they have a different save model
• Show them there, but updated the implementation to support the write command if they are modified

[DonLakeFlyer]

Maybe include update to PARAM_VALUE to include: new field: persistence_type - defaults to 0 (for back compatibility), set to 1 to indicate param is not no-save model. Requires PARAM_WRITE_PESISTENT call. Then you have generic no-save support for params. As opposed to magic assocated with MAV_COMP_ID_ESC . Pretty it up with enums and what not.

[dakejahl]

Extending PARAM_VALUE is probably a non-starter I'm afraid. I'm hoping some of the other people I tagged will chime in

[julianoes]

We can add it to the wishlist for param v3 that I'm advocating for in mavlink/mavlink#2344.

[hamishwillee]

I wasn't aware of this command, that should work. We'd need to add MAV_COMP_ID_ESC to the MAV_COMPONENT enum.

Nope, you'd add a MAV_TYPE for ESC. You could also add an ID to reserve it, but you shouldn't need to.

[hamishwillee]

Where are we with this? I mean @dakejahl do you have a way forward?

Extending PARAM_VALUE is probably a non-starter I'm afraid. I'm hoping some of the other people I tagged will chime in

Why not? There is nothing magic about extending messages as long as we don't care if this works on MAVLink 1 systems.

But I agree with @julianoes in mavlink/mavlink#2344 - not that we necessarily "need" a new protocol, but that we should at least start with a set of requirements and agree explicitly how we can or cannot deliver them by extending the current protocol

[dakejahl]

I wasn't aware of this command, that should work. We'd need to add MAV_COMP_ID_ESC to the MAV_COMPONENT enum.

Nope, you'd add a MAV_TYPE for ESC. You could also add an ID to reserve it, but you shouldn't need to.

The component ID is necessary if we want to use MAV_CMD_PREFLIGHT_STORAGE to specifically target the ESC component to do the save. We don't want the side effect of telling the FC to save.

Where are we with this? I mean @dakejahl do you have a way forward?

I've already implemented Option 1 in both QGC and PX4 and it's working well. There are a few finishing touches left with the QGC UI but otherwise I'm just waiting on mavlink. I'd like to present this at the dev summit 2 months from now.

Why not? There is nothing magic about extending messages as long as we don't care if this works on MAVLink 1 systems.

In that case I would also suggest the PARAM_VALUE contain the error enum from mavlink/mavlink#2344. I just figure it is harder to extend an existing message(s) than adding something new.

[hamishwillee]

The component ID is necessary if we want to use MAV_CMD_PREFLIGHT_STORAGE to specifically target the ESC component to do the save. We don't want the side effect of telling the FC to save.

Yuck - that is not how it is supposed to work.

A component is supposed to emit its type - say "MAV_TYPE_ESC" and from that you would work out the component ID of the ESC component you want to target.
If you are suggesting that the ESC is actually inside the FC and you want to use the component id as a filter for the FC to redirect the message that is a misuse of the id.

For this I'd suggest we consider using another param in https://mavlink.io/en/messages/common.html#MAV_CMD_PREFLIGHT_STORAGE to identify a specific component managed by the flight stack.

Where are we with this? I mean @dakejahl do you have a way forward?

I've already implemented Option 1 in both QGC and PX4 and it's working well. There are a few finishing touches left with the QGC UI but otherwise I'm just waiting on mavlink. I'd like to present this at the dev summit 2 months from now.

So you don't want to do the param thing but instead want to use AM32_EEPROM? OK, I'll add to the dev call next week to see if we can get more eyes on this.

Why not? There is nothing magic about extending messages as long as we don't care if this works on MAVLink 1 systems.

In that case I would also suggest the PARAM_VALUE contain the error enum from mavlink/mavlink#2344. I just figure it is harder to extend an existing message(s) than adding something new.

So your proposal is that instead of emitting a PARAM_ERROR you include the error in the PARAM_VALUE? What's the benefit for of that approach over just emitting the PARAM_ERROR (sorry if you answered this in the thread - I'm skim reading because the backlog is too big).

It is usually easier to extend a message than add a new one - I'll add a note asking if this would be better.

[IamPete1]

Why not tunnel the pass-through connection over MAVLink? This is how AP does DroneCAN, This has the advantage that MAVLink does not have to have any knowledge of how the ESC protocols work. The native ESC tools will be more upto-date and have better support than a native MAVLink interface built into a GCS.

[dakejahl]

@hamishwillee

So you don't want to do the param thing but instead want to use AM32_EEPROM? OK, I'll add to the dev call next week to see if we can get more eyes on this.

No not necessarily, I would actually prefer something that is not AM32 specific and could be extended for other ESCs. In mavlink/mavlink#2324 I implemented the message as opaque data and the general consensus seemed to suggest using TUNNEL instead. I want to formalize this in mavlink though, it's an important enough thing that I think it deserves a detected set of messages.

So your proposal is that instead of emitting a PARAM_ERROR you include the error in the PARAM_VALUE? What's the benefit for of that approach over just emitting the PARAM_ERROR (sorry if you answered this in the thread - I'm skim reading because the backlog is too big).

The benefit would be a single response path -- ie you don't look for both PARAM_VALUE and a potential PARAM_ERROR. It greatly simplifies the state management in the GCS.

@IamPete1

Why not tunnel the pass-through connection over MAVLink? This is how AP does DroneCAN, This has the advantage that MAVLink does not have to have any knowledge of how the ESC protocols work. The native ESC tools will be more up to-date and have better support than a native MAVLink interface built into a GCS.

The pass-through connection is MSP/four-way protocol, this involves putting the ESCs into bootloader mode and bitbanging serial over the motor signal wire. Recently AM32 introduced dshot programming mode, which allows you to write settings using dshot commands. The goal with the mavlink design is to be protocol agnostic, such that the FC can choose to implement either four-way or dshot programming mode. If we made the mavlink design even more generic we could support any ESC with any protocol. The core problem the mavlink design is trying to solve is the transmission of key/value settings for read/write.

The native ESC tools will be more upto-date and have better support than a native MAVLink interface built into a GCS.

Yeah this is a good point and I've considered it. The problem is we would either need to add mavlink to the ESC tool (AM32-configurator) or we'd need to have some kind of tool that runs on the side that opens a virtual com port for AM32-configurator to connect to which then packs the data into a mavlink tunnel message. We would still end up needing to maintain that code though.

[IamPete1]

Any thoughts on ESC firmware update? Something that MAVLink tunnelling may be able to do, I guess it depends if there any baud rate changes or timing critical stuff.

I suspect adding native MAVLink to the AM32 configurator would be quite easy. Maybe you have seen the web based DroneCAN tool, https://can.ardupilot.org/, it does CAN over MAVLink. However, it might be harder for other ESC firmware's, especially those with closed source tools.

I guess there is no reason we can't support tunnelling and the proposed settings protocol.

[dakejahl]

Any thoughts on ESC firmware update?

Yeah I was thinking mavlink FTP to upload the file to the SD card and a new MAV_CMD to trigger the fw update. The command.param1 would specify the ESC to update (or all).

Maybe you have seen the web based DroneCAN tool, https://can.ardupilot.org/

I have not, thanks for sharing I'll check it out

[jsm09a]

I'm running BLHeli_32 - and to update the ESC configs, I've been fishing out a separate FC running Betaflight and using that to run the ESC configurator. Not sure if the "tunnelling" mentioned above would be sufficient for running the ESC configurator through PX4, but that sure would be nice/appreciated !

[hamishwillee]

Had a small chat about this in the MAV call. As I understand this, ArduPilot put the flight stack into a pass through mode on reboot, after which you can configure the ESC using configurator tools from other vendors.
The benefit is that you're not reliant on the stability of the configuration tool, or a particular firmware or whatever for your ESC - which might change.

We're not so sure ardupilot would update, and that might be reasonable given that this is something you only need to do once with an ESC.

We do get that this would be nice to have, if the interface you are using is stable.

1. Can we get some information about the stability of the underlying mechanisms you're planning on using?

[dakejahl]

The benefit is that you're not reliant on the stability of the configuration tool, or a particular firmware or whatever for your ESC - which might change.

Yes of course, if the flight controller has an exposed USB port and the firmware is capable of ESC passthrough then that is for sure the best way to do it. I'm not arguing that point at all. This is solving a totally different problem which I explain at the top of the page of the RFC.

Can we get some information about the stability of the underlying mechanisms you're planning on using?

It's not deployed so therefore it's not stable. It's really not a hard thing to understand though:

1. AM32 defines an "eeprom" which is just a block of memory on the ESC. This "eeprom" consists of 48 bytes where each byte corresponds to a configurable setting. Most of these settings are user configurable while some are read-only.

2. The second byte of this block is the eeprom_version which indicates the eeprom layout. All the GCS needs to do is check this eeprom_version and use the appropriate schema. The schema can be contained in the GCS source code and can be updated as new versions of AM32 are released. Maybe in the future we could contribute to AM32 to automatically publish the schema for releases and have QGC automatically download it at runtime, thus allowing an old QGC to function with a newer AM32 release.

3. Mavlink development dialect is for development. We can add it there and if for some unseen reason this totally fails down the road, we remove it. There is an implementation in QGC and PX4 tested and ready to go.

There are a lot of users who have been asking about this. I think we as engineers significantly underappreciate how painful working with hardware can be for those with less experience. Many people have no idea their ESC firmware needs to be configured until they've already built their airframe and bolted and soldered everything into place. Some firmware doesn't support passthrough. Some flight controllers don't have USB.

[DonLakeFlyer]

I think we as engineers significantly underappreciate how painful working with hardware can be for those with less experience.

Truer things have not been said. Need to be able to make hardware morons like me be able to understand what must be done :)

[hamishwillee]

@dakejahl So to be very clear, you're saying that the block format may not be stable but that the GCS (@DonLakeFlyer) is willing to do the work to maintain it - right?

But what about the protocol that you're using to query the blocks in the first place - how stable is that?

It is hard to get things into common that ardupilot think they will never take. That said, I think this is something most users would prefer than the passthrough thing, so they might change their minds. I'd be OK with a PR into development so people can start testing in mainline PX4. But it shouldn't go into a release until its agreed it can move into common.xml.

[dakejahl]

@dakejahl So to be very clear, you're saying that the block format may not be stable but that the GCS (@DonLakeFlyer) is willing to do the work to maintain it - right?

Define stable. To me stable means it has been tested over a long period of time and is unlikely to change. Given this is not even in the specification yet it is by my definition unstable. I doubt it will change though. And no, I will do the work to maintain it.

But what about the protocol that you're using to query the blocks in the first place - how stable is that?

Might be easiest if I share the link to the PR code directly 😀
mavlink/mavlink@master...dakejahl:mavlink:pr-am32_eeprom

Read
The MAV_CMD_AM32_REQUEST_EEPROM is sent from the GCS to request the block of data. The FC emits AM32_EEPROM message which contains the data.

Write
If the GCS wants to change any one of the settings the GCS will emit the AM32_EEPROM message with the corresponding bits in the write_mask set indicating the value should be written.

I'd be OK with a PR into development so people can start testing in mainline PX4. But it shouldn't go into a release until its agreed it can move into common.xml.

Agreed. We will ship PX4 builds with development.xml and tell our users to use the QGC daily build in order to leverage this feature.

[dagar]

stable means it has been tested over a long period of time and is unlikely to change

Stable means unchanging, and in this particular case it would just need to some basic version checking so if future ESCs do change in some unexpected way we can at least prevent the user some shooting themselves in the foot (or alert them to the risk).

MAV_CMD_AM32_REQUEST_EEPROM, AM32_EEPROM, etc

This might be more palatable to a wider audience as a generic EEPROM request and value (targeted at an ESC).

[dakejahl]

This might be more palatable to a wider audience as a generic EEPROM request and value (targeted at an ESC).

I tend to agree but I'm not familiar with how other ESCs expose their settings, It was my original idea with mavlink/mavlink#2324

I'll make a post in the Bluejay Discord and see if anyone has interest.

[peterbarker]

Stepping back a bit here.

Both PX4 and ArduPilot struggle fitting everything they want to fit into the flight controller's flash memory.

Taking up flash with code which is only used when setting a vehicle up on the ground seems a poor path to go down.

Note that we do carry such "dead code" already in ArduPilot with our "blheli passthrough stuff".

ArduPilot's various methods of dealing with ESC configuration (CAN-over-mavlink, blheli-passthrough) have worked extremely well in the former and kind-of-OK in the latter.

Speaking of which. How does the betaflight family of autopilots solve the problem being addressed in this PR?

[dakejahl]

Both PX4 and ArduPilot struggle fitting everything they want to fit into the flight controller's flash memory.

That's why I pushed so hard for making this a passthrough style mechanism. The FC only needs to query the data and send it down over mavlink and vice versa. I'm certain this method consumes less flash than a full passthrough implementation.

Speaking of which. How does the betaflight family of autopilots solve the problem being addressed in this PR?

Betaflight doesn't have a means to remotely update the ESC configuration. All betaflight FCs have an exposed USB port and generally betaflight systems are small FPVs with USB easily accessible. PX4 and Ardupilot systems are often big and tightly integrated, the USB port is often blocked by a case, carbon plate, or some other piece of hardware.

We sell a lot of ESCs and a lot of Jetson carrier boards. About once a week someone complains that they can't update their ESC with the Jetson PAB carrier. You could say our design is bad for not exposing the USB port for this purpose, but there's nothing we can do about that now. In the future we will likely mux the port to allow both direct USB to Jetson and exposed USB. But that's only part of the problem. Other users have fully integrated their FC and ESC before even testing (sure we could blame them for not testing) and then discover when they go to takeoff that their motor KV value is wrong and needs to be changed. I've experienced this firsthand, wasting 4 hours helping a customer disassemble and reassemble the aircraft the night before a demo so that we could update the ESC motor KV... like I said above we really need to empathize with our users who don't know any better. Anything we can do to make setting up these systems easier is gold.

[hamishwillee]

So

• @dagar rejects a more generic solution in ESC_RAW_SETTINGS message mavlink#2324 (comment) but proposes that it would be more palatable in ESC_RAW_SETTINGS message mavlink#2324 (comment)
  • FWIW It would be to me too. Right now it sounds like AM32 has very widespread use, but if we can make this completely generic that would future proof it.
• @julianoes suggests that a tunnel would be a valid alternative to a generic protocol in ESC_RAW_SETTINGS message mavlink#2324 (comment)
• @peterbarker Doesn't want dead code for something done only once, and @dakejahl responds that this passthrough approach is probably lighter weight than the full passthrough approach ArduPilot uses.
• @DonLakeFlyer and @dakejahl argue that this (either generic or AM32 specific) would make workflows much easier for users.
• @hamishwillee is happy for this to be trialed in development.xml. Not happy for anything in development.xml to go into a released PX4. Ever. Perhaps it is time for px4.xml or qgc.xml to allow this IFF agreed by Don, Daniel, Jake.

I'm not seeing consensus. The main blocker seems to be that ArduPilot has a solution and don't want to carry another one.

@peterbarker What I'm hearing from Don is that it is a complete pain. Users are important too - and IMO I personally would never use the ArduPilot solution if this alternative existed.
So let's at least not preclude ArduPilot taking it in future:

• If you were interested in taking this into ArduPilot, what approach would you use and why?

I'd suggest that whatever solution we decide on is:

• Easy to add or remove from firmware
• Easy to detect support for (by GCS) - allow it to hide the UI
• Includes mechanism to know when the protocol is not working - i.e. the underlying transport protocol broke or is not known, or if the specific block format is unknown - allowing the UI to decide to hide on that basis.
• My preference is that it be generic as in original proposal.

I'll make a post in the Bluejay Discord and see if anyone has interest.

@dakejahl Did you get a response?

[dakejahl]

Yes I did get a response from the maintainer of Bluejay. Very helpful information.
https://discord.com/channels/822952715944460368/1433504604028670074/1433529398476279920
However he said he has no experience with mavlink/qgc/px4. There's no interest to do the work.

I'm also not personally interested in fully implementing/testing/supporting other ESC firmwares. But I think structuring the mavlink message design to support other firmware down the road is prudent.

Bluejay uses 4-way -- the same as blheli and am32. This is the protocol to talk to the bootloader to program the ESCs. This is how the "ESC Passthrough" works as well. And technically Ardupilot could use their existing pass-through implementation to program the ESCs, they'd just need to create a small software layer to convert the data from mavlink to four-way. In the same way that they have the software layer to convert from MSP to four-way (MSP over USB from the Configurator app on the PC, to four-way over the motor signal wires to the ESC bootloader)

If you were interested in taking this into ArduPilot, what approach would you use and why?

👍👀

[DronecodeBot]

This pull request has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/am-32-firmware-and-px4/45776/9