Fix/versioning dropdown

[cotti]

Closes #2018, may solve a portion of #1660

Taking into account feedback and study cases provided by @shainaraskas, this PR handles the following:

Versioning system inference priority refinements

Pages in docs-content now rely primarily on frontmatter metadata before legacy page matching.

• If there is at least one direct product reference, use the first one
• If there is a product or deployment applicability set, use the first match, starting from product
• Fallback to legacy page mapping and further existing methods

Question: Should this be applied to all repositories from the get-go, or should we phase this change?

Dropdown rendering

If a page has a versioning system that doesn't warrant version selection, don't render it.

[Mpdreamz]

We indeed should have one rule for all repositories.

The rules should degrade from most explicit to least.

product in frontmatter > product in applicable_to frontmatter > product in docset.yml > legacy url mapping > docset git repository name.

We should warn if frontmatter defines both product and applicable to product. Perhaps we can even get rid of product in frontmatter alltogether? cc @shainaraskas @florent-leborgne

[Mpdreamz]

Maybe we should have an exhaustive ProductFromApplicability and have a unit test to ensure all known products are handled.

[shainaraskas]

We should warn if frontmatter defines both product and applicable to product.

Are you talking about the following (product>id>cloud-kubernetes and applies_to>deployment>eck)?

---
applies_to:
  deployment:
    eck: ga
products:
  - id: cloud-kubernetes
---

or are we talking about the product node of applies_to? or when they both exist, such as in this case?

---
navigation_title: EDOT .NET
description: The Elastic Distribution of OpenTelemetry (EDOT) .NET provides an extension of the OpenTelemetry SDK for .NET, configured for the best experience with Elastic Observability. 
applies_to:
  stack:
  serverless:
    observability:
  product:
    edot_dotnet: ga
products:
  - id: cloud-serverless
  - id: observability
  - id: edot-sdk
---

we MOSTLY have both of these pieces of information on a page. they don't do the same thing, so I don't think we can deduplicate them.

[shainaraskas]

Pages in docs-content now rely primarily on frontmatter metadata before legacy page matching.

• If there is at least one direct product reference, use the first one
• If there is a product or deployment applicability set, use the first match, starting from product
• Fallback to legacy page mapping and further existing methods
  Question: Should this be applied to all repositories from the get-go, or should we phase this change?

I'd like to see this ship in docs-content first to see if it is doing what we want. I'm reviewing this totally blind and unable to see where it might break down.

I'd suggest that we keep the legacy page mapping as the "primary" piece of information, as a lot of times we manually ordered that list to have the dropdown work the way we wanted. this only doesn't work in places were we have no mapped pages to fall back on, or where a mapping belongs to an unversioned product. I'd personally probably pick:

• mapped pages
• products maybe?
• applies_to (product -> stack -> deployment -> serverless)

[florent-leborgne]

2 cents, maybe that's too simplistic but my current impression is that we're trying to build a complex logic based on unreliable data:

• applies_to can have 0 to many values
• products can have 0 to many values
• legacy mapping can have 0 to many URLs from different old guides/sources

Wouldn't it be simpler to have a specific frontmatter metadata that determines the versioning system of a page/set of page instead of relying on existing metadata initially built for a different purpose to try to guess it?

We could do something like

• if new_metadata isn't set then use stack versioning
• set new_metadata in docset or at page level if impossible at docset level to a different one (none, ece, eck, somethingelse)

Then look if we can map something to previous versions within the set versioning system. At least the setting is explicit and when a page isn't showing what it should, we just need to set it right explicitly rather than tweak the logic/play with priorities on other metadata.

That way, we could also explicitly choose more easily to not show any version selection on certain doc sets based on their very purpose (like fundamentals, or some getting started pages)

The simpler the solution is, the more writers can actually understand, control and predict such behaviors without needing a lead or docs-eng.

[Mpdreamz]

We need these rules to have all pages be able default to a versioning scheme/product. We can leverage it more places than just showing the version picker (e.g indexing to ES).

1. product in frontmatter
2. product in applicable_to frontmatter
3. product in docset.yml
4. legacy url mapping (Treated as legacy).
5. docset git repository name.

Rule number 2 is the preferred way to be explicit, we always ask folks to include an applies_to section if appropriate.

We need rule number 1 to provide an override but we should keep the tagging at a minimum if applies_to already suffices.

We need option 4 for as long as docs-content has pages that would not be covered by rules 1,2,3
We need option 5 when onboarding new documentation sets. Smaller projects need not care about this at all.

In the case of multiple products:

• 2 > pick the first non stack product
• 2 > Error if no unique product (error message says make it explicit by setting product in frontmatter)
• N > Error if no unique product (error message says make it explicit by setting product in frontmatter).

This may be less simplistic to implement but it puts a lot less cognitive overhead for most documentation sets.

New projects just have to ensure assembler.yml, products.yml and versioning.yml is correct and don't need to tag every page with something.

[shainaraskas]

@Mpdreamz we might want to chat about this synchronously tomorrow, because a lot of the terms that you're using aren't 100% clear to me (we say product a lot but it might multiple things in our metadata). I also think we differ in opinion about how reliable these data points are, and what to prioritize. I worry that, if I am understanding your order, we'll be breaking more dropdowns than we fix with this change because of how that products frontmatter was initially populated.