Skip to content

HIVE-29096: Enhance the metatool updateLocation for multiple table or… #5989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dengzhhu653
Copy link
Member

@dengzhhu653 dengzhhu653 commented Jul 21, 2025

… serde properties

What changes were proposed in this pull request?

Enhance the metatool updateLocation to accept multiple properties

Why are the changes needed?

Sometimes we need to update multiple properties in case of NameNode switch. For example for Iceberg tables, both of the metadata_location and previous_metadata_location should update

Does this PR introduce any user-facing change?

Yes

How was this patch tested?

The new added UT,
Running the example in the jira.

Copy link

@Aggarwal-Raghav
Copy link
Contributor

@dengzhhu653, for the iceberg usecase mentioned in description, I believe updating location will not fully resolve the problem as metadata (snapshot.avro, metadata.json etc) also needs to be updated (unless there is some internal solution to update the metadata) explicitly.

Reference: apache/iceberg#1617 , I think we need to implement this proceduce in hive
https://iceberg.apache.org/docs/latest/spark-procedures/#rewrite_table_path
CC @deniskuzZ

But this PR will be helpful post updating the metadata in iceberg.

@dengzhhu653
Copy link
Member Author

@dengzhhu653, for the iceberg usecase mentioned in description, I believe updating location will not fully resolve the problem as metadata (snapshot.avro, metadata.json etc) also needs to be updated (unless there is some internal solution to update the metadata) explicitly.

Reference: apache/iceberg#1617 , I think we need to implement this proceduce in hive https://iceberg.apache.org/docs/latest/spark-procedures/#rewrite_table_path CC @deniskuzZ

But this PR will be helpful post updating the metadata in iceberg.

Thank you @Aggarwal-Raghav for pointing it out. Not sure if the rewrite_table_path will update the metadata_location in HMS table, looks it's not, as it uses the register_table to load the new copied table in the last step.

@deniskuzZ
Copy link
Member

deniskuzZ commented Jul 25, 2025

@Aggarwal-Raghav yes, you are right, that won't work for iceberg, at least not until apache/iceberg#13141 is resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants