DL duplicates data when using pointers to structs

[Tisten]

I've created a test for this in #151

I could try to solve it myself, but I'm not sure how to best put a tag on some data in JSON so a pointer can refer to it. Any opinions?

[lundmark]

From my PoV I'd appreciate it if they were separate json-objects serialized just like the actual data would look like in memory but as json instead. Then have the structs refer to them by index or something similar?

I guess there should be a storage passed in the serialization context that handles all existing references to serialized objects to handle this?

[Tisten]

So all structs which someone points to will get a magic "__index" member which pointers can refer to?

[lundmark]

Something like that would make sense imo yes, I'm not sure what @wc-duck thinks tho?

[wc-duck]

hmm... in that case we could just use any name, such as "__ref_id" or similar so that if you write it by hand you can just write something that makes sense.
It could always be generated as "my_type_1" or similar.

But also, this would only be needed if you reference an item that is also part of an array or struct right? Otherwise they are just put in
'__subdata' :
{
'__item_1' : {}
}

Right? Long time since I worked with the pointers? :)

[Tisten]

Yes, that is right. I agree that writing something sensible when authoring it by hand is a great feature. And keeping pointer payloads which aren't used as arrays or members of structs is also fine, it is one correct way of doing it, and the closest to what is there today.

For improved json readability I could also imagine it working by putting each pointer payload in the place where it is first referred to, and then all other pointers to it will just refer to the "__ref_id". But that is a matter of taste, both would work and option 1 is probably least work.

[Tisten]

F.Y.I:
I have continued developing datalibrary based on our internal needs ever since 2022, and this issue is now solved using the "__ref_id" idea.
It also solved the "writing pointer payloads 'inline' " part of #147
I wont be able to make a pull request of the solution since it builds on the already created pull requests, and around 100 commits after those PRs are done. In my own fork I've also ditched 32 bit support and strict C compatibility, so it is no longer possible to simply merge the code with your original repository.
That said, I would love to make our alternative version available as well, so if someone want to make the effort to merge something, then at least it could be done.