API: Add support to copy object logical mode #9500
Conversation
Make a shallow copy of an object, copy metadata only. Limited to the same repository and branch. Experimental.
nopcoder
added
area/API
|
📚 Documentation preview at https://pr-9500.docs-lakefs-preview.io/ (Updated: 9/9/2025, 2:46:00 PM - Commit: 6c92dbd) |
| if srcRepository != destRepository { | ||
| return nil, fmt.Errorf("%w: clone must be between the same repository", graveler.ErrInvalid) | ||
| } | ||
| if srcRef != destBranch { | ||
| return nil, fmt.Errorf("%w: clone must be between the same branch", graveler.ErrInvalid) | ||
| } |
There was a problem hiding this comment.
I think this should default to the CopyEntry behaviour
| dstEntry := *srcEntry | ||
| dstEntry.Path = destPath | ||
| dstEntry.CreationDate = time.Now() | ||
| err = c.CreateEntry(ctx, destRepository, destBranch, dstEntry, opts...) |
There was a problem hiding this comment.
You're not checking if you're allowed to use the same physical address! There are possible races with GC!
There was a problem hiding this comment.
What are the conditions to allowed? this version does not take responsibility and it can cause issues if the caller will cause this race with GC.
It assume that the caller will use this API on uncommitted data.
| @@ -2765,6 +2765,29 @@ func (c *Catalog) PrepareGCUncommitted(ctx context.Context, repositoryID string, | |||
| }, nil | |||
| } | |||
|
|
|||
| func (c *Catalog) CloneEntry(ctx context.Context, srcRepository, srcRef, srcPath, destRepository, destBranch, destPath string, opts ...graveler.SetOptionsFunc) (*DBEntry, error) { | |||
There was a problem hiding this comment.
Why not change CopyEntry behavior and create CloneEntry?
There was a problem hiding this comment.
The GC grace time is associated to the physical object creation. Here I'm not accessing the underlaying storage and I can't verify based on the entry if it is a valid 'clone' candidate.
This is why the additional 'mode' where the caller is responsible in the clone - assume that lakeFSFS will create clones while the data is uncommitted on the same branch.
There was a problem hiding this comment.
@itaiad200 do you have a suggestion when to verify that the copy is a safe clone on lakeFS?
| expectedError: graveler.ErrNotFound, | ||
| }, | ||
| { | ||
| name: "successful_clone_with_metadata", |
Co-authored-by: guy-har <[email protected]>
Co-authored-by: guy-har <[email protected]>
Make a shallow copy of an object, copy metadata only.
Limited to the same repository and branch.
Experimental
Closes #9499