feat: iterate entries to sync from LMDB store (FOR-139)

[jyeshe]

What/Why

This PR is a complement for #440.

It enables the sync from a LMDB store to any other by allowing to iterate through out all LMDB key values (visiting all the branches and leaves for all hashpath trees).

Additional notes

It depends on twilson63/elmdb-rs#12

[jyeshe]

1> FromLmdbStore = #{~"store-module" => hb_store_lmdb, ~"name" => ~"cache-mainnet", ~"resolve" => false}.
#{<<"name">> => <<"cache-mainnet">>,<<"resolve">> => false,
  <<"store-module">> => hb_store_lmdb}
2> ToLmdbStore2 = #{~"store-module" => hb_store_lmdb, ~"name" => ~"cache-mainnet-lmdb2"}.
#{<<"name">> => <<"cache-mainnet-lmdb2">>,
  <<"store-module">> => hb_store_lmdb}
3> hb_store:sync(FromLmdbStore, ToLmdbStore2).
ok
4> f(FromLmdbStore). FromLmdbStore = #{~"store-module" => hb_store_lmdb, ~"name" => ~"cache-mainnet"}.
ok
5> FromLmdbStore = #{~"store-module" => hb_store_lmdb, ~"name" => ~"cache-mainnet"}.
#{<<"name">> => <<"cache-mainnet">>,
  <<"store-module">> => hb_store_lmdb}
6> {ok, FromEntries} = hb_store:list(FromLmdbStore, ~"/"), length(FromEntries).
173
7> Groups = [X || X <- FromEntries, composite == hb_store:type(ToLmdbStore2, X)], length(Groups).
173
8> Paths = [<<G/binary, "/", F/binary>> || G <- Groups, F <- element(2, hb_store:list(ToLmdbStore2, G))], hd(Paths).
<<"-UNEqPQpf9GS-rI2lMblUgHFaWOTbEnC_sCV00DY6WE/name">>
9> SimplePaths = [P || P <- Paths, hb_store:type(ToLmdbStore2, P) == simple], length(SimplePaths).
1406
10> [SP || SP <- SimplePaths, hb_store:read(ToLmdbStore2, SP) =/= hb_store:read(FromLmdbStore, SP)].
[]
11> ProcAssignments = ~"assignments/yegjj-WALfaZLP2sqSdpmWJu6aLk1C2pYiPBUv2Ns2s".
<<"assignments/yegjj-WALfaZLP2sqSdpmWJu6aLk1C2pYiPBUv2Ns2s">>
12> Slots = [<<ProcAssignments/binary, "/", S/binary>> || S <- element(2, hb_store:list(ToLmdbStore2, ProcAssignments))], length(Slots).
6
13> SlotListings = [<<S/binary, "/", L/binary>> || S <- Slots, L <- element(2, hb_store:list(ToLmdbStore2, S))], hd(SlotListings).
<<"assignments/yegjj-WALfaZLP2sqSdpmWJu6aLk1C2pYiPBUv2Ns2s/0/cursor">>
14> [SL || SL <- SlotListings, hb_store:read(ToLmdbStore2, SL) =/= hb_store:read(FromLmdbStore, SL)].
[]

[samcamwilliams]

I think the need for this arrises more from a default config issue than the code? hb_opts is currently defining cache-mainnet/lmdb as the default, then a fallback FS store at cache-mainnet IIRC. Better to change that config (FS store => cache-mainnet/fs?) than to add code here.

[jyeshe]

done

[samcamwilliams]

Any reason to change the LMDB Rust side from :list to match_prefix? When I first read this I assumed we meant matching the value with a prefix, which could be much more expensive?

[jyeshe]

hey @samcamwilliams , the reason for the match_prefix and the new KV with group: prefix on the key is because we need a way to index the groups to prevent iterating the whole database to get the groups.
It follows this approach in order to satisfy the requirement to have a generic sync based on hb_store behaviour.
If we could have more specialized implementations we could adopt another strategy like iterating all the keys from source store and copying them to the destination store.

[jyeshe]

removed match_prefix in favor of the iterator/fold

[samcamwilliams]

Why make another new key? Is there a way around this? The line 442 is already a bit of a nuisance (the existence of ID/keys in the DB implies that ID is a group), but is passable. Adding 2 extra keys to the DB per group will add complexity long-term, I think.

[jyeshe]

I have left it to allow a single call for hb_store:type on groups. But as we've just discussed, we intend to apply a global iterator approach. cc @twilson63

[jyeshe]

removed once we can get the next process with the iterator

[samcamwilliams]

Left some comments!

[jyeshe]

Left some comments!

@samcamwilliams This gonna be helpful to merge hydration results from different push machines and to cleanup/reset certain processes. I have addressed all the comments and merged to the base branch for139/store-sync.