feat: sync from an hb store to another (FOR-139)

[jyeshe]

What

Copies the entries from one store to another using hb_store behaviour (in a store agnostic way).

Note on scope: this implementation was tested only on local stores, more specifically syncing from hb_store_fs to hb_store_lmdb

Why

Use cases from FOR-139:

• Migrating data between different store implementations
• Creating backups of store data
• Synchronizing stores across different environments

How to use

FromStore = #{~"store-module" => hb_store_fs, ~"name" => ~"cache-mainnet", ~"resolve" => false}.
ToStore = #{~"store-module" => hb_store_lmdb, ~"name" => ~"cache-mainnet"}.
hb_store:sync(FromStore, ToStore).

Validation steps

1. Comment the ?DEFAULT_PRIMARY_STORE in hb_opts:default_message
2. rm -rf cache-*
3. Start HyperBEAM and hydrate a process (e.g. S5iWl7L76dBpvrm-i_SRWODchdLYBGwYeSrIxC8v-vI)
4. Stop HyperBEAM
5. Uncomment ?DEFAULT_PRIMARY_STORE and restart HyperBEAM
6. Run the commands below (where resolve=false is used to allow syncing links):

1> FromFsStore = #{~"store-module" => hb_store_fs, ~"name" => ~"cache-mainnet", ~"resolve" => false}.
#{<<"name">> => <<"cache-mainnet">>,<<"resolve">> => false,
  <<"store-module">> => hb_store_fs}
2> ToLmdbStore = #{~"store-module" => hb_store_lmdb, ~"name" => ~"cache-mainnet"}.
#{<<"name">> => <<"cache-mainnet">>,
  <<"store-module">> => hb_store_lmdb}
3> hb_store:sync(FromFsStore, ToLmdbStore).
ok
4> FsStore = #{~"store-module" => hb_store_fs, ~"name" => ~"cache-mainnet"}.
#{<<"name">> => <<"cache-mainnet">>,
  <<"store-module">> => hb_store_fs}
5> {ok, FsEntries} = hb_store:list(FsStore, "/"), length(FsEntries).
35934
6> Groups = [X || X <- FsEntries, composite == hb_store:type(ToLmdbStore, X)], length(Groups).
35931
7> NotGroups = [X || X <- FsEntries, composite =/= hb_store:type(ToLmdbStore, X)], length(NotGroups).
3
8> Paths = [<<G/binary, "/", F/binary>> || G <- Groups, F <- element(2, hb_store:list(ToLmdbStore, G))], hd(Paths).
<<"lY1jQ7wKCS10hTAsbbValARN1TzFEf4glHdnl3eSFtE/name">>
9> SimplePaths = [P || P <- Paths, hb_store:type(ToLmdbStore, P) == simple], length(SimplePaths).
263456
10> [SP || SP <- SimplePaths, hb_store:read(ToLmdbStore, SP) =/= hb_store:read(FsStore, SP)].
[] 
11> ProcAssignments = ~"assignments/S5iWl7L76dBpvrm-i_SRWODchdLYBGwYeSrIxC8v-vI/".
12> Slots = [<<ProcAssignments/binary, S/binary>> || S <- element(2, hb_store:list(ToLmdbStore, ProcAssignments))], length(Slots).
1151
13> {hd(Slots), lists:last(Slots), hb_store:type(ToLmdbStore, <<ProcAssignments/binary, "1150">>)}.
{<<"assignments/S5iWl7L76dBpvrm-i_SRWODchdLYBGwYeSrIxC8v-vI/0">>,
 <<"assignments/S5iWl7L76dBpvrm-i_SRWODchdLYBGwYeSrIxC8v-vI/999">>,
 composite}
14> SlotListings = [<<S/binary, "/", L/binary>> || S <- Slots, L <- element(2, hb_store:list(ToLmdbStore, S))], hd(SlotListings).
<<"assignments/S5iWl7L76dBpvrm-i_SRWODchdLYBGwYeSrIxC8v-vI/0/ao-types">>
15> [SL || SL <- SlotListings, hb_store:read(ToLmdbStore, SL) =/= hb_store:read(FsStore, SL)].
[]

Addtional Notes

TBD:

• How to deal with retries when sync returns {error, {sync_failed, FailedKeyValues}}
• How to sync stores in different nodes/machines

[jyeshe]

Reading nested paths (e.g slots entries) was not working because link targets are returned with a leading /.
In this print we can see the slot 0 from hydrated process S5iWl7L76dBpvrm-i_SRWODchdLYBGwYeSrIxC8v-vI was not_found while it was pointing to a entry having a leading "/".

[jyeshe]

Formatter agent merit

[jyeshe]

% rebar3 eunit --module=hb_store_lmdb
...
All 17 tests passed.
% rebar3 eunit --module=hb_store
...
All 32 tests passed.