-
Notifications
You must be signed in to change notification settings - Fork 53
GH-765: Do not close/free imported BaseStruct objects #766
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This comment has been minimized.
This comment has been minimized.
#763 (comment) suggests adding new or overloaded methods that do not call close instead of introducing a breaking change. Is there any preference wrt naming these? |
Perhaps by analogy with the base library, |
Just FYI, I'm experimenting a bit, and neither the
The overload of |
Hmm. A single boolean flag seems fine for Alternatively: add a constructor boolean parameter for |
I've pushed a second attempt that introduces overloaded import methods with a boolean argument. I've moved all the close calls to the Data class for clarity. |
ArrowArrayStreamReader reader = new ArrowArrayStreamReader(allocator, stream); | ||
if (closeImportedStructs) { | ||
stream.close(); | ||
} | ||
return reader; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(1) the indentation seems off?
(2) should we put this in a try-with-resources to at least fix the edge case noted (failure to free in case of exception)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indentation fixed has been corrected.
Regarding the stream#close
call, I went for maintaining the exact current behavior. The ArrowArrayStreamReader
constructor did not call close in a finally block. Let me know if you would prefer to change this.
Looks reasonable to me, thanks. |
checkstyle is still unhappy:
|
Additionally, could we add a small unit test if possible? Otherwise LGTM |
Checkstyle issues should be fixed. I'll add some unit tests next. |
18b3aff
to
6c093db
Compare
…orted BaseStruct objects
I've added a couple of tests. |
What's Changed
This PR removes the direct and indirect calls to
BaseStruct#close
fromorg.apache.arrow.c.Data
. By not eagerly closing/freeing these objects callers can reuse instances multiple times.This contains breaking changes.: a quick check in, for instance,org.apache.arrow.c.StreamTest
shows that a lot of existing code is already written using try-with-resources. This change will not have any impact there. Code that does not use a try-with-resources or try-finally-close pattern and instead counts onData
to callclose
will report memory leaks after this change.The second version of this PR removes the breaking change.
Closes #765.