Support k-NN index for exact search in semantic text field

[fen-qin]

Description

If a user has multiple vector fields in a KNN index currently, and wants to perform exact search on certain fields, this k-NN index will allow users to not create graphs on those fields and can do exact search and approximate search at the same time.
This PR will support the kNN exact search index to semantic text field.

• k-NN index PR: Add index parameter to remove unnecessary vector data structure creation k-NN#2768
• feature branch: feature/exact-search-new-experience (based on 3.2)

Test

• create a multiple vector fields kNN index with semantic type

PUT /my-nlp-index_multiple
{
    "settings":  {
        "index.knn": true 
    },
    "mappings":  {
        "properties":  {
            "id":  {
                "type": "text" 
            },
            "passage_text01":  {
                "type": "semantic",
                "model_id": "{your_model_id}",
                "dense_embedding_config":  {
                    "index": true,
                    "method":  {
                        "engine": "lucene",
                        "name": "hnsw",
                        "parameters":  {
                        }
                    }
                }
            },
            "passage_text02":  {
                "type": "semantic",
                "model_id": "{your_model_id}",
                "dense_embedding_config":  {
                    "index": false 
                }
            }
        }
    }
}

• get the index mapping

GET /my-nlp-index_multiple/_mapping
{
	"my-nlp-index_multiple": {
		"mappings": {
			"properties": {
				"id": {
					"type": "text"
				},
				"passage_text01": {
					"type": "semantic",
					"model_id": "{your_model_id}",
					"raw_field_type": "text",
					"dense_embedding_config": {
						"method": {
							"engine": "lucene",
							"name": "hnsw",
							"parameters": {}
						},
						"index": true
					}
				},
				"passage_text01_semantic_info": {
					"properties": {
						"embedding": {
							"type": "knn_vector",
							"dimension": 768,
							"method": {
								"engine": "lucene",
								"space_type": "l2",
								"name": "hnsw",
								"parameters": {}
							}
						},
						"model": {
							"properties": {
								"id": {
									"type": "text",
									"index": false
								},
								"name": {
									"type": "text",
									"index": false
								},
								"type": {
									"type": "text",
									"index": false
								}
							}
						}
					}
				},
				"passage_text02": {
					"type": "semantic",
					"model_id": "CmN6h5kB1prYUG0Dabxz",
					"raw_field_type": "text",
					"dense_embedding_config": {
						"index": false
					}
				},
				"passage_text02_semantic_info": {
					"properties": {
						"embedding": {
							"type": "knn_vector",
							"dimension": 768,
							"index": false
						},
						"model": {
							"properties": {
								"id": {
									"type": "text",
									"index": false
								},
								"name": {
									"type": "text",
									"index": false
								},
								"type": {
									"type": "text",
									"index": false
								}
							}
						}
					}
				}
			}
		}
	}
}

Related Issues

Resolves #1470

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[heemin32]

1. This can be applied to sparse embedding as well?
2. Target branch should be main not 3.2.

[fen-qin]

1. This can be applied to sparse embedding as well?
2. Target branch should be main not 3.2.

yes. @heemin32 k-NN PR is pending in the feature branch, which is a rebase from 3.2, we probably need to merge to a feature branch as well