Skip to content

Conversation

mridula-s109
Copy link
Contributor

@mridula-s109 mridula-s109 commented Sep 15, 2025

🚨 Please check with search inference team on when to merge this - we're working out some rate limiting issues ahead of GA of ELSER on EIS

Summary

Implements dynamic default selection for semantic_text fields to automatically use ELSER on EIS
(.elser-2-elastic) when available, with graceful fallback to ML nodes (.elser-2-elasticsearch).

Problem

Currently, semantic_text fields are hardcoded to default to .elser-2-elasticsearch (ML nodes). This misses
opportunities for better performance and cost efficiency when EIS is available.

Solution

  • Dynamic Detection: Uses ModelRegistry.containsDefaultConfigId() to detect EIS availability
  • Smart Fallback: Automatically selects .elser-2-elastic when available, falls back to
    .elser-2-elasticsearch
  • Zero Configuration: Works transparently without requiring user changes
  • Full Compatibility: On-prem users continue using ML nodes until they enable cloud-connected mode

Changes

  • Added getPreferredElserInferenceId() method for dynamic selection logic
  • Updated semantic_text field mapper to use dynamic default instead of hardcoded value
  • Added comprehensive tests for both EIS available/unavailable scenarios

Testing

  • ✅ All existing tests pass (backward compatibility verified)
  • ✅ New test covers dynamic selection logic
  • ✅ Manual testing confirms proper fallback behavior

Impact

  • Cloud users: Automatically get better performance with EIS
  • On-prem users: No changes, continue using ML nodes seamlessly
  • Existing fields: Completely unaffected, no migration needed

@mridula-s109
Copy link
Contributor Author

Hey @ioanatia! 👋

Just implemented the dynamic ELSER default selection for semantic_text fields.

The approach is intentionally minimal -leverages existing ModelRegistry.containsDefaultConfigId() for
EIS detection rather than building new infrastructure. New semantic_text fields automatically default
to .elser-2-elastic when available, gracefully fall back to .elser-2-elasticsearch when not.

A few things I'd especially appreciate feedback on:

  • Does the overall approach make sense?
  • Is the error handling sufficient (graceful fallback when ModelRegistry fails)?
  • Any concerns about the scope or missing edge cases?

Still draft mode, but wanted to get your input before finalizing. Thanks!

@ioanatia
Copy link
Contributor

relying on modelRegistry.containsDefaultConfigId(EIS_ELSER_INFERENCE_ID) seems to be right.

but I think we need more tests.
Maybe take a look at some of the tests from https://github.com/elastic/elasticsearch/tree/main/x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/elastic and see if we can mock the presence of EIS in tests.
Then we can properly test that we pick the right inference ID - by first creating an index with a semantic_text field (and unspecified inference ID) and then requesting the mapping of this new index. We should see the default EIS endpoint.

@ioanatia
Copy link
Contributor

In terms of tests, what we can also do is to add another mock service in https://github.com/elastic/elasticsearch/tree/main/x-pack/plugin/inference/qa/test-service-plugin/src/main/java/org/elasticsearch/xpack/inference/mock

and then use it in yaml tests.

the InferenceService allows to register default endpoints:

default List<DefaultConfigId> defaultConfigIds() {

I think we can mock the default endpoints we have in EIS and use the same name in the mock inference service.
Let me know if you need any help!

@elasticsearchmachine elasticsearchmachine added Team:Search - Relevance The Search organization Search Relevance team and removed needs:triage Requires assignment of a team area label Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Oct 7, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @mridula-s109, I've created a changelog YAML for you.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/search-relevance (Team:Search - Relevance)

@seanhandley
Copy link

Thanks @mridula-s109 !

Can we merge this around Oct 22-24? This would mean it goes live on serverless Monday 27, ECH in 9.3, and fits our rollout plan ☺️

cc @maxjakob for visibility

Copy link
Contributor

@Mikep86 Mikep86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking better! I left some comments about how to clean up the tests. IMO the biggest thing left is the default inference ID assertion in InferenceSemanticTextIT. If that is truly returning variable values, it could be an indicator of something we need to address. Or it could be a race condition in a flaky test 😁 . Best to characterize it and address it early.

@seanhandley
Copy link

FYI folks we're working on some rate limiting issues with ELSER and we may want to defer this.

Please check with @maxjakob or myself before merging.

@mridula-s109
Copy link
Contributor Author

FYI folks we're working on some rate limiting issues with ELSER and we may want to defer this.

Please check with @maxjakob or myself before merging.

Thanks for letting me know @seanhandley , will do the same!

@mridula-s109
Copy link
Contributor Author

FYI folks we're working on some rate limiting issues with ELSER and we may want to defer this.

Please check with @maxjakob or myself before merging.

Thanks @Mikep86! All comments addressed. Have verified about the default assertion in the SemanticTextEISDefaultIT, there is no flakiness or underlying issue at the moment. Since we're holding off on merging due to the ELSER rate limiting work, no rush on further review. Take your time! 😊

*
* My understanding is that the @Before will be run after the node starts up and wouldn't be sufficient to handle
* this scenario. That is why this needs to be @BeforeClass.
*/
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was initially unsure about keeping this comment due to potential redundancy, but decided to leave it as-is since the added verbosity makes it more explanatory.

@mridula-s109
Copy link
Contributor Author

@ioanatia @Mikep86 quick update: I’ve addressed the review feedback and pushed the latest changes.
Just a heads-up we’ve scheduled a short sync with @seanhandley and @maxjakob on Wednesday, October 22 (1:00–1:15 PM) to proceed with merging this PR.
Please let me know if there’s anything else you’d like me to adjust before then. 🙌

Copy link
Contributor

@Mikep86 Mikep86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM to merge once we get the 👍 from @seanhandley and @maxjakob

Comment on lines +25 to +30
@Before
public void setUp() throws Exception {
super.setUp();
// Ensure the mock EIS server has an authorized response ready before each test
mockEISServer.enqueueAuthorizeAllModelsResponse();
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my understanding of the conversation in #134708 (comment) (and the linked references), I think we only need the @BeforeClass annotated method. However, this additional @Before method will be harmless at worst, so nothing to block over. As @ioanatia said earlier, we can sync with the ML team and clean this up later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement :SearchOrg/Relevance Label for the Search (solution/org) Relevance team Team:Search - Relevance The Search organization Search Relevance team v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants