Configuring Federated Catalog Crawling in MVD

[mohamedkhalilbeldi]

Dear All,

I observed that in this Minimum Viable Dataspace (MVD), there is a provider catalog (not a federated catalog that aggregates assets from other catalogs), and the assets have been injected using seeding.

I plan to use the Eclipse EDC Federated Catalog and seek guidance on how to declare other catalogs in the federated catalog so they can be crawled. From the Samples/federated-catalog, the target nodes are statically added, either with a node resolver extension or files that contain participants and not the catalog server. Therefore, I’m asking:

Which API should I use to configure and register other catalogs for crawling when the Federated Catalog is already running? In a production setup, adding a new catalog or node wouldn't be done by adding static nodes, building, and rerunning.

How is the Federated Catalog authenticated for other servers' catalogs?

Any insights would be greatly appreciated.

[paullatzelsperger]

The Dataspace Protocol (DSP) defines a catalog endpoint, which is the only place where catalogs can be obtained.
All the FC does is take a list of TargetNodes (from the TargetNodeDirectory), perform those DSP catalog requests against them and cache the results. As a convenience feature, the FC offers an API endpoint where this consolidated list of catalogs can be obtained.

Which API should I use to configure and register other catalogs for crawling when the Federated Catalog is already running?

The TargetNodeDirectory offers a way to add another TargetNode at runtime. All you have to do is implement an API endpoint in an extension, that uses this method

How is the Federated Catalog authenticated for other servers' catalogs?

Whatever authentication mechanism is configured in DSP, for example the Decentralized Claims Protocol (DCP).

[mohamedkhalilbeldi]

Thank you for your response.

I am trying to add a TargetNode to the Federated Catalog that I am setting up with the Minimum Viable Dataspace (MVD). To do this, I used a Federated Catalog (FC) example from the Eclipse EDC Samples repository, and I used fixed-node-resolver to implement the TargetNode.getAll() method.

However, I am facing an issue regarding the structure of the TargetNode in a DID environment like MVD. I attempted to use the following TargetNode:

TargetNode participantNode = new TargetNode( "https://w3id.org/edc/v0.0.1/ns/", "provider-fc", "http://localhost:8092/api/dsp", protocolList );

In this case, http://localhost:8092 is the address of the provider's catalog in the MVD setup, which I want to add as a node to my Federated Catalog (FC). Unfortunately, this configuration did not work as expected. I received a 400 Bad Request error from the Federated Catalog, and on the provider catalog side, I encountered the error: 'The given ID must conform to did:method:identifier but did not.' Even after changing the ID to a DID structure, it still didn’t work."

Could you provide any insights into the correct structure for defining a TargetNode in this context (DID)? Am I missing any required configurations or dependencies?

Any help would be greatly appreciated!

[Moritz72]

I believe the ID of the TargetNode (the one that is currently "provider-fc") should be the DID of the participant to which the catalog at http://localhost:8092/api/dsp belongs to.