[EPIC] Transaction Support Issues and Pull Requests

[jonathanc-n]

What's the feature are you trying to implement?

There is an accumulation of issues + PRs related to transactions which has made it quite difficult to follow on to the transaction support progress (there are duplicate pull requests, issues, etc. being made ). This is a list to help new contributors be aware of what is being worked on + reviewers have a easy place to access pull requests that can be looked at 😄:

Issues (without pull request)

• Write Support (EPIC): [EPIC] Iceberg-rust Write support #700
• Metadata Tables (EPIC): [EPIC] feat: support metadata tables #823
• Row Delta (row level modifications + concurrency): Support RowDeltaAction #1104
• Integration Tests for Transaction: Add integration tests for transaction APIs #1322
• Retrying Commits: Support commit retrie #964
• Union by Name for Schema: Update a TableSchema from a Schema #698
• Initial Default: Implement initial-default #737 (waiting for MergeAppend)
• Validate in SnapshotProduceAction: Support validate and related logic in Snapshot #1344

Pull Requests 🚀

• Position Delete Writer: feat: support position delete writer #704
• Support Entries Metadata Table: feat: Support metadata table "Entries"  #863 (currently blocking metadata table support)
• MergeAppend: feat: support merge append action #902
• Scan Delete Support: Scan Delete Support Part 4: Delete File Loading; Skeleton for Processing #982 + Scan Delete Support Part 5: Positional Delete Parsing #1011 + Scan Delete Support Part 6: Equality Delete Parsing #1017
• Refine Record Batches: feat: refine record batch project #1014
• Partition Splitter: Partition Writer Support Part 1: add partition splitter #1040
• Partition Stats: feat: Support PartitionStats #1111
• Update Schema API: feat: Add basic operations for UpdateSchema #1172
• PartitionMetricsEvaluator: (build up towards Row Delta): feat: Add basic support for PartitionAndMetricsEvaluator #1189
• Rewrite Manifest Action: feat: support rewrite manifest action #1237
• Branch Update Actions: feat: Support branch actions using ManageSnapshots #1298
• Name Mapping: feat: Add IndexByName and IndexById to Namemapping #1299
• Retrieving Latest Table (refresh): Support retrieving the latest Iceberg table on table scan #1297
• Updating Summary Properties: feat: Introduce snapshot summary properties #1336
• Update Table in Catalog: feat: sql catalog support update table #862
• Update statistics: feat(transaction): Add update statistics #1359
• Snapshot validation: feat(transaction): Support snapshot validation #1353

I have listed out some issues that are dependant on other pull requests. I will be maintaining this as issues + pull requests are being made, feel free to let me know if I need to add something!

[Xuanwo]

Thank you @jonathanc-n for gathering this information. I’ll begin reviewing the pending PRs.

[liurenjie1024]

Thanks @jonathanc-n for this summary. Transaction support is a somewhat large topic. Except reviewing prs, we also need to have a design to split it into smaller issues, so that the community could make progress on it. I'm do the planning recently.