feat(coredns): distinguish between mutliple owners

[farodin91]

What does it do ?

Idea is to distinguish records between multiple clusters under coredns

Motivation

I want to host a service with the same domain in two clusters.

More

• Yes, this PR title follows Conventional Commits
• Yes, I added unit tests
• Yes, I updated end user documentation accordingly

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mloiseleur for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @farodin91. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[farodin91]

@ivankatliarchuk Hi, I'm not sure if this is going in the right direction. But probably you have a better idea?

At moment, it isn't possible as findEp merges endpoints by different owners.

[ivankatliarchuk]

I dunno how the infrastructure looks like. Implementing this with just external-dns is not a viable option. You are missing a coordinator or some proper service-discovery solution.

graph TD;
    A(Coordinator Service) --> B(CoreDNS);
    A --> C(external-dns Cluster A);
    A --> D(external-dns Cluster B);

Loading

Where coordinator/decider could be a custom webhook or whatever service.

[ivankatliarchuk]

Or something similar to

where each cluster external-dns writes to it's own clusters coredns if needed, and then a service does required merge of data based on business requirments.

Basically external-dns is not a solution, but just a part of it.

[farodin91]

My current setup works fine most of the time.
I run multiple external DNS services across multiple clusters against a single etcd. I built a custom version of CoreDNS based on the webhook provider, which filters out entries with the owner from different cluster. However, it seems that the owner text isn't added to all A records anymore. Therefore, it works for old cases, but not new ones. Code: https://github.com/GDATASoftwareAG/external-dns-coredns-webhook/blob/main/coredns.go#L253

Essentially, I added a coordinator that works like a proxy and filters out entries based on owners.

The current idea is to use setIdentifier to distinguish records.

Another idea is to add a custom identifier, which could easily be added as an opt-in, and which would be stored in the etcd. This would allow us to pre-filter records based on the identifier. This should work in simple cases, but complex cases could require something different. (I haven't had any use cases for it yet).

Could any of my ideas become part of the core provider?

[farodin91]

All my setup as in diagram

graph TD;
    A(external-dns Cluster A) --> AP(external-dns Proxy A with owner filter);
    AP--> C(ETCD);
    B(external-dns Cluster B) --> BP(external-dns Proxy B with owner filter);
    BP --> C; 
    C --> D(CoreDNS);

Loading

[farodin91]

Here is my proxy idea: GDATASoftwareAG/external-dns-coredns-webhook#24

[ivankatliarchuk]

Hard to say. I think in this case, is better to decouple coordinator/merger service from external-dns completely.

This is an open source project without any short or long term strategy. I do not think there are any plans to support multiple owners, just a hypotetical proposal #5619 (comment)

I'm not sure of your specific use case or environment—whether it's a home lab or a commercial project—but I'd generally recommend creating a decision record. This would allow you to compare solutions like ExternalDNS+wrappers/webhooks with other options, such as a fully custom approach, service discovery, or a service mesh, to weigh the pros and cons.

[farodin91]

@ivankatliarchuk Thank you for your answers. I think a coordinator or merger service would be too complex for our solution.

I really like my idea at GDATASoftwareAG/external-dns-coredns-webhook#24, which involves pre-filtering endpoints before they are used in external-dns (i.e. no merging of the same DNS entries, and only working with managed entries by a single external-dns).
Do you think such a solution could be merged with the offical CoreDNS provider?
PS: One benefit of pre-filtering is that it reduces memory consumption, as only a small proportion of the entries need to be handled.