Crash bug with latest level

[ralphtheninja]

System Information

• Aedes: 0.42.5
• NodeJS: 14.12.0
• OS: Ubuntu 20.04 LTS
• Arch: x86_64

Describe the bug
aedes-persistence-level crashes with latest level.

To Reproduce
Steps to reproduce the behavior:

I've created a repository that reproduces the crash here: https://github.com/ralphtheninja/aedes-crash-with-level
How to reproduce is described in the readme.

Expected behavior
It shouldn't crash.

Additional context
It only seems to happen when you publish a qos: 2 message.

[robertsLando]

I think that there could be some breaking change in a major level release, could you send a PR for this? Level persistence hasn't been updated for a long time

[ralphtheninja]

I think that there could be some breaking change in a major level release, could you send a PR for this? Level persistence hasn't been updated for a long time

I could. But it would be nice to know what the problem is first and where to apply the fix. Maybe there's a workaround as well?

[ralphtheninja]

@vweevers You thought it had something to do with multiple encodings?

[robertsLando]

@ralphtheninja It seems something related to: kawanet/msgpack-lite#81

[robertsLando]

The error is in this line: https://github.com/moscajs/aedes-persistence-level/blob/master/persistence.js#L398

[robertsLando]

You should try to print blob to see the content of blob when it crash

[robertsLando]

You could also give a try to: https://github.com/JoshuaWise/tiny-msgpack

[ralphtheninja]

The blob content looks like:

key is incoming:mqttjs_d634c29b:51471

[robertsLando]

@ralphtheninja Please try to add a log also here so we also see the packet before the encode

[ralphtheninja]

key: incoming:mqttjs_9bd12a52:29811
newp: Packet {
  cmd: 'publish',
  brokerId: undefined,
  brokerCounter: 0,
  topic: 'foo',
  payload: <Buffer 62 61 72>,
  qos: 2,
  retain: false,
  messageId: 29811
}

[ralphtheninja]

It seems we store a Buffer here https://github.com/moscajs/aedes-persistence-level/blob/master/persistence.js#L387 (what msgpack.encode() returns), but when we get the value back it's a string.

[ralphtheninja]

If I pass { valueEncoding: 'binary' } to level() before passing it into aedes-persistence-level it works. This is a good workaround. Maybe we should update the docs, saying something about using latest level needs to have valueEncoding set to 'binary'.

[ralphtheninja]

It also works if I pass { valueEncoding: 'binary' } to .get() here https://github.com/moscajs/aedes-persistence-level/blob/master/persistence.js#L392