[ExtraHop][Qualys GAV] Cannot execute ILM policy delete step

[janvi-elastic]

Kibana/Elasticsearch Stack version: 8.18.0

Describe the bug:

The kibana_system role lacks the necessary permissions to delete system indices related to logs-extrahop.investigation and logs-qualys_gav.asset, as defined in the ILM policy located here.

Steps to reproduce:

1. Checkout the sharadcrest:package-extrahop-investigation-datastream branch for ExtraHop package and janvi-elastic:package-qualys_gav branch for Qualys GAV package and create a zip of the respective package.
2. Upload the package zip to a hosted deployment.
3. Add the integration.
4. Monitor the hidden index under Stack Management > Index Management and wait for the ILM policy’s delete phase to trigger.

Current behavior:

• It shows permission issue in deleting the index

For Qualys GAV:

{
  "failed_step": "delete",
  "step_info": {
    "type": "security_exception",
    "reason": "action [indices:admin/delete] is unauthorized for user [found-internal-kibana4-server] with effective roles [found-internal-kibana4-server,kibana_system] on indices [.ds-logs-qualys_gav.asset-default-2025.07.24-000001], this action is granted by the index privileges [delete_index,manage,all]"
  }
}

For ExtraHop:

{
  "failed_step": "delete",
  "step_info": {
    "type": "security_exception",
    "reason": "action [indices:admin/delete] is unauthorized for user [found-internal-kibana4-server] with effective roles [found-internal-kibana4-server, kibana_system] on indices [.ds-logs-extrahop.investigation-default-2025.07.23-000001], this action is granted by the index privileges [delete_index, manage, all]"
  }
}

Expected behavior:

• Index must be delete after the time duration mentioned in the ILM policy

[gmarouli]

Hi @janvi-elastic , ILM inherits the user permissions and roles, here it looks that the policy was created by found-internal-kibana4-server that does not have privileges to delete indices.

Can you create a user with the correct privileges to set up the ILM policy?

I believe that ILM works as expected, and the problem lies in the configuration, if there are no objections shall we close this issue?

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[ShourieG]

@gmarouli, We opened this ticket cause it seemed like a similar issue to this and even in the Kibana channel in this thread, we were advised to add this new index to elasticsearch index role descriptor here.

In the slack thread we mentioned that we created users with all the required privileges but kibana-system was not inheriting them for some reason. This was the process which was triggering the ILM but since this is a system level user we were manually unable to modify any of it's configs.

[gmarouli]

Thanks for the extra information @ShourieG , this looks exactly the same, so there is a clear path forward. I will remove the ILM tag and let you take it from there on.

Do you belong to a team for which we have a label so I can triage this issue to your team?

[ShourieG]

Thanks for the extra information @ShourieG , this looks exactly the same, so there is a clear path forward. I will remove the ILM tag and let you take it from there on.

Do you belong to a team for which we have a label so I can triage this issue to your team?

@gmarouli, I'm part of Security-Service-Integrations @elastic/security-service-integrations , but there's no direct tag for this here.

[PeteGillinElastic]

I'm going to assign the Authorization label, since that seems to be most commonly used for changes to KibanaOwnedReservedRoleDescriptors (and that is, IIUC, what's required to fix this). It's not quite right because it's not really an issue with the authorization bug per se, just a configuration thing. But it'll make make the needs:triage label go away... Apologies to the security team for the spam.