Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
503 commits
Select commit Hold shift + click to select a range
915e992
arrow-select: Implement concat for `RunArray`s (#7487)
brancz May 13, 2025
8dbca1e
Speed up arithmetic kernels, reduce `unsafe` usage (#7493)
Dandandan May 13, 2025
1f15130
Add `arrow_reader_clickbench` benchmark (#7470)
alamb May 13, 2025
4770573
Update arrow_reader_row_filter benchmark to reflect ClickBench distri…
alamb May 15, 2025
21ce2c1
Improve documentation for Parquet `WriterProperties` (#7491)
alamb May 15, 2025
741121b
Update documentation for ParquetReader (#7501)
alamb May 15, 2025
bf63e2a
Improve `Field` docs, add missing `Field::set_*` methods (#7497)
alamb May 15, 2025
847534d
Fix Clippy in CI for Rust 1.87 release (#7514)
alamb May 16, 2025
1a5999a
Simplify ParquetRecordBatchReader control logic (#7512)
alamb May 16, 2025
7bab215
arrow: add concat structs benchmark (#7520)
asubiotto May 16, 2025
c37f5a0
Minor: Add examples to ProjectionMask documentation (#7523)
alamb May 19, 2025
6721ec1
docs: fix typo for Decimal128Array (#7525)
burmecia May 19, 2025
ce18e5b
Introduce `ReadPlan` to encapsulate the calculation of what parquet r…
alamb May 19, 2025
5a278c6
RecordBatchDecoder: skip RecordBatch validation when `skip_validation…
nilskch May 19, 2025
3c45926
Make parquet-show-bloom-filter work with integer typed columns (#7529)
adamreeve May 20, 2025
31ea844
chore(deps): update criterion requirement from 0.5 to 0.6 (#7527)
mbrobbel May 20, 2025
0b75873
Create version of LexicographicalComparator that compares fixed numbe…
Dandandan May 20, 2025
d6a2351
Fix record API support for reading INT32 encoded TIME_MILLIS (#7511)
njaremko May 20, 2025
7261909
Minor: Add a parquet row_filter test, reduce some test boiler plate (…
alamb May 20, 2025
45bda04
Refactor `build_array_reader` into a struct (#7521)
alamb May 20, 2025
36b2a27
Support Utf8View for Avro (#7434)
kumarlokesh May 21, 2025
0d774fe
arrow-select: add support for optimized concatenation of struct array…
asubiotto May 22, 2025
e9df239
Fix `filter_record_batch` panics with empty struct array (#7539)
thorfour May 22, 2025
05e0d15
Add Map support to arrow-avro (#7451)
jecsand838 May 23, 2025
531370f
Fix CI on main due to logical conflict (#7542)
alamb May 23, 2025
1c380b8
fix: Panic in pretty_format function when displaying DurationSecondsA…
zhuqi-lucas May 26, 2025
88d59f8
Prevent FlightSQL server panics for `do_put` when stream is empty or …
superserious-dev May 26, 2025
7c60194
Add support for creating random Decimal128 and Decimal256 arrays (#7427)
Weijun-H May 26, 2025
0da003b
Improve error messages if schema hint mismatches with parquet schema …
alamb May 26, 2025
72c7294
Fix `Result` name collision in parquet_derive (#7548)
jspaezp May 27, 2025
aac69b5
Clarify Docs: NullBuffer::len is in bits (#7556)
alamb May 28, 2025
0a4ffa5
Fix: Converted feature flight-sql-experimental to flight-sql (#7546)
kunalsinghdadhwal May 28, 2025
2436cd5
correct docs for set_column_index_truncate_length (#7553)
etseidl May 28, 2025
7e85b48
Adding Bloom Filter Position argument in parquet-rewrite (#7550)
JigaoLuo May 28, 2025
71b4b0f
Minor: remove filter code deprecated in 2023 (#7554)
alamb May 30, 2025
307d768
Use apache/arrow-{go,java,js} in integration test (#7561)
kou May 31, 2025
b397fa2
Minor: Enable byte view for clickbench benchmark (#7565)
zhuqi-lucas May 31, 2025
66903b6
Use PR title and description for commit message (#7563)
kou May 31, 2025
0f414f4
add references for defaults in WriterProperties (#7558)
etseidl May 31, 2025
d17dce0
Fixed a warning build build: function never used. (#7577)
JigaoLuo May 31, 2025
28c3cb9
Initial API for reading Variant data and metadata (#7535)
mkarbo May 31, 2025
3d88c11
Revert "Minor: remove filter code deprecated in 2023 (#7554)" (#7583)
alamb Jun 1, 2025
8a77bce
Improve the PR template for usable as commit message (#7586)
kou Jun 2, 2025
b10567f
Fix GenericBinaryArray docstring. (#7588)
brunal Jun 2, 2025
e5fb84a
[benches/row_format] fix typo in array lengths (#7572)
ctsk Jun 2, 2025
3681540
Add FixedSizeBinary to `take_kernel` benchmark (#7592)
alamb Jun 2, 2025
0ae9f66
Truncate Parquet page data page statistics (#7555)
etseidl Jun 3, 2025
e814b97
fix: error reading multiple batches of `Dict(_, FixedSizeBinary(_))` …
albertlockett Jun 3, 2025
950f4d0
Add a strong_count method to Buffer (#7569)
westonpace Jun 3, 2025
6deefb7
Add CI tests for `parquet-variant` crate and fix clippy (#7601)
alamb Jun 4, 2025
f92ff18
Add `coalesce` kernel and`BatchCoalescer` for statefully combining se…
alamb Jun 5, 2025
ef91857
arrow-select: add support for merging primitive dictionary values (#7…
asubiotto Jun 5, 2025
026356b
Include footer key metadata when writing encrypted Parquet with a pla…
rok Jun 6, 2025
9e575bd
feat: add `row_group_is_[max/min]_value_exact` to StatisticsConverter…
CookiePieWw Jun 6, 2025
9ead0d1
Add concatenate kernel benchmark for `StringViewArray` (#7617)
alamb Jun 6, 2025
44d7194
Improve coalesce_kernel benchmark to capture inline vs non inline vie…
alamb Jun 6, 2025
7739a83
Improve `coalesce` and `concat` performance for views (#7614)
Dandandan Jun 7, 2025
da461c8
Revert "Improve `coalesce` and `concat` performance for views (#7614)…
Dandandan Jun 7, 2025
52d8d56
Revert "Revert "Improve `coalesce` and `concat` performance for views…
Dandandan Jun 8, 2025
9d172a8
Adding Encoding argument in `parquet-rewrite` (#7576)
JigaoLuo Jun 8, 2025
23e18bc
Improve `coalesce` kernel tests (#7626)
alamb Jun 9, 2025
312e2fd
Move variant interop test to Rust integration test (#7602)
alamb Jun 9, 2025
375bee7
[Variant] Add commented out primitive test casees (#7631)
alamb Jun 9, 2025
8d4beae
Optimize length calculation in row encoding for fixed-length columns …
ctsk Jun 9, 2025
05363f6
feat: add AsyncArrowWriter::into_inner (#7604)
jpopesculian Jun 9, 2025
9482f78
[array] Remove unwrap checks from GenericByteArray::value_unchecked (…
ctsk Jun 9, 2025
13fc3c8
Fix the error info of `StructArray::try_new` (#7634)
xudong963 Jun 10, 2025
7211502
feat: support append_nulls on additional builders (#7606)
albertlockett Jun 10, 2025
857614c
Fix reading encrypted Parquet pages when using the page index (#7633)
adamreeve Jun 11, 2025
04300b4
Minor: Remove outdated FIXME from `ParquetMetaDataReader` (#7635)
etseidl Jun 11, 2025
3fe458e
Minor: Add version to deprecation notice for `ParquetMetaDataReader::…
etseidl Jun 11, 2025
2be261b
Deprecate old Parquet page index parsing functions (#7640)
etseidl Jun 11, 2025
e5ad232
Update FlightSQL `GetDbSchemas` and `GetTables` schemas to fully matc…
sgrebnov Jun 11, 2025
8d6cd76
Ensure page encoding statistics are written to Parquet file (#7643)
etseidl Jun 12, 2025
e32f545
Use approximate comparisons for pow tests (#7646)
adamreeve Jun 12, 2025
71ee9d9
[Variant] Implement read support for remaining primitive types (#7644)
superserious-dev Jun 13, 2025
2f2e705
feat: add constructor to help efficiently upgrade key for GenericByte…
albertlockett Jun 13, 2025
c87a4d9
Add `pretty_format_batches_with_schema` function (#7642)
lewiszlw Jun 13, 2025
1029974
chore: group prost dependabot updates (#7659)
mbrobbel Jun 13, 2025
58b34cb
Remove `lazy_static` dependency (#7669)
Expyron Jun 16, 2025
f48efc2
chore(dependabot): update all Cargo manifests (#7672)
mbrobbel Jun 16, 2025
3a15f84
[Variant] Simplify creation of Variants from metadata and value (#7663)
alamb Jun 16, 2025
639b5bb
chore(dependabot): explicitly include root workspace and arrow-pyarro…
mbrobbel Jun 16, 2025
f5f09ea
Finish implementing Variant::Object and Variant::List (#7666)
scovich Jun 17, 2025
e1ade7b
chore: fix a typo in `ExtensionType::supports_data_type` docs (#7682)
mbrobbel Jun 17, 2025
a19fc62
Add `BatchCoalescer::push_filtered_batch` and docs (#7652)
alamb Jun 17, 2025
e6c93c0
Add `RecordBatch::schema_metadata_mut` and `Field::metadata_mut` (#7664)
emilk Jun 17, 2025
3837ac0
arrow-row: Add support for REE (#7649)
brancz Jun 17, 2025
f37b114
Document REE row format and add some more tests (#7680)
alamb Jun 17, 2025
ed25bba
Implement Array Decoding in arrow-avro (#7559)
jecsand838 Jun 17, 2025
56ac4dc
Initial Builder API for Creating Variant Values (#7653)
PinkCrow007 Jun 17, 2025
6227419
Speedup `interleave_views` (4-7x faster) (#7695)
Dandandan Jun 18, 2025
20c1c34
Make variant iterators safely infallible (#7704)
scovich Jun 19, 2025
fe65b8d
[Variant] Add variant docs and examples (#7661)
alamb Jun 19, 2025
ecd2905
arrow-data: Add REE support for `build_extend` and `build_extend_null…
brancz Jun 20, 2025
75008eb
feat: add min max aggregate support for FixedSizeBinary (#7675)
alexwilcoxson-rel Jun 20, 2025
7276819
Split out variant code into several new sub-modules (#7717)
scovich Jun 20, 2025
1bed04c
Optimize coalesce kernel for StringView (10-50% faster) (#7650)
alamb Jun 20, 2025
fbaf7ce
Support write to buffer api for SerializedFileWriter (#7714)
zhuqi-lucas Jun 20, 2025
469c7ee
Define a "arrow-pyrarrow" crate to implement the "pyarrow" feature. (…
brunal Jun 20, 2025
7b374b9
arrow-array: Implement PartialEq for RunArray (#7727)
brancz Jun 21, 2025
1ededfe
[Variant] Introduce new type over &str for ShortString (#7718)
friendlymatthew Jun 21, 2025
e54b72b
fix: Do not add null buffer for `NullArray` in MutableArrayData (#7726)
comphead Jun 22, 2025
2788762
fix JSON decoder error checking for UTF16 / surrogate parsing panic (…
nicklan Jun 22, 2025
df702cf
Prepare for `55.2.0` release (#7722)
alamb Jun 22, 2025
6d01dd9
Perf: Optimize comparison kernels for inlined views (#7731)
zhuqi-lucas Jun 23, 2025
b1ea729
arrow-row: Refactor arrow-row REE roundtrip tests (#7729)
brancz Jun 23, 2025
4549ced
Allow per-column parquet dictionary page size limit (#7724)
XiangpengHao Jun 23, 2025
ce69739
fix: Implement support for appending Object and List variants in Vari…
Weijun-H Jun 23, 2025
a795030
[Variant] Use `BTreeMap` for `VariantBuilder.dict` and `ObjectBuilder…
friendlymatthew Jun 24, 2025
2b40d1d
[Variant] Add Variant::as_object and Variant::as_list (#7755)
alamb Jun 24, 2025
a49ce3e
Add testing section to pull request template (#7749)
alamb Jun 24, 2025
121371c
feat: [Variant] Add Validation for Variant Deciaml (#7738)
Weijun-H Jun 24, 2025
8d8541c
[Variant] Support nested lists and object lists (#7740)
friendlymatthew Jun 24, 2025
389b2b0
[Variant] Fix several overflow panic risks for 32-bit arch (#7752)
scovich Jun 24, 2025
b6240b3
Respect `PARQUET_TEST_DATA` in variant_interop test (#7747)
alamb Jun 24, 2025
71ac9bd
Extend the fast path in GenericByteViewArray::is_eq for comparing aga…
jhorstmann Jun 25, 2025
0366140
Variant: Write Variant Values as JSON (#7670)
carpecodeum Jun 25, 2025
4d3906c
Add fallible versions of temporal functions that may panic (#7737)
adriangb Jun 25, 2025
10d9714
Introduce `MAX_INLINE_VIEW_LEN` constant for string/byte views (#7719)
alamb Jun 25, 2025
340c7dc
[Variant] Improve write API in `Variant::Object` (#7741)
friendlymatthew Jun 25, 2025
d7fc416
Support `FixedSizeList` RowConverter (#7705)
findepi Jun 25, 2025
7d3a25a
[Variant] Support creating nested objects and object with lists (#7778)
friendlymatthew Jun 25, 2025
d6c421c
[VARIANT] Validate precision in VariantDecimalXX structs and add miss…
scovich Jun 25, 2025
b269422
[Variant] Check pending before `VariantObject::insert` (#7786)
friendlymatthew Jun 26, 2025
01c5efc
Add sort_kernel benchmark for StringViewArray case (#7787)
zhuqi-lucas Jun 26, 2025
0de463e
Add benchmark for about view array slice (#7781)
ctsk Jun 26, 2025
1c397a9
Add specialized coalesce path for PrimitiveArrays (#7772)
alamb Jun 26, 2025
72e91fc
[Variant] Consolidate examples for json writing (#7782)
alamb Jun 26, 2025
f8bcc58
[VARIANT] impl Display for VariantDecimalXX (#7785)
scovich Jun 26, 2025
5505113
Fix clippy for Rust 1.88 release (#7797)
alamb Jun 27, 2025
1fdb318
[Variant] Improve getter API for `VariantList` and `VariantObject` (#…
friendlymatthew Jun 27, 2025
e8017f7
Minor: fix clippy after logical conflict (#7803)
alamb Jun 27, 2025
5ff4167
Update base64 requirement from 0.21 to 0.22 (#7791)
dependabot[bot] Jun 27, 2025
e42df82
[Variant] Simplify `Builder` buffer operations (#7795)
friendlymatthew Jun 27, 2025
8fd870b
Minor: Remove filter code deprecated in 2023 (try 2) (#7584)
alamb Jun 27, 2025
06cbc33
Change default parquet statistics truncation to be 64 bytes (#7578)
alamb Jun 27, 2025
e2162cc
Remove old `flight-sql-experimental` feature flag (#7551)
alamb Jun 27, 2025
e930492
Update to parquet-format 2.11.0 (#7570)
etseidl Jun 27, 2025
67f7e99
parquet_derive: update in working example for ParquetRecordWriter (#7…
LanHikari22 Jun 27, 2025
452397f
Add `TimeMilli` and `TimeMicro` fields and conversions for the record…
njaremko Jun 27, 2025
d910a1d
feat: add constructor to efficiently upgrade dict key type to remaini…
albertlockett Jun 27, 2025
c316974
Stop writing statistics to Parquet page headers by default, add optio…
etseidl Jun 27, 2025
be338f9
Implement Type and Array for Decimal32 and Decimal64 (#7098)
CurtHagenlocher Jun 27, 2025
2754ce5
Add schema with only primitive arrays to `coalesce_kernel` benchmark …
alamb Jun 27, 2025
92ae675
chore(deps)!: update pyo3 requirement from 0.24.1 to 0.25.1 (#7505)
mbrobbel Jun 27, 2025
8b4a90e
Add `arrow-pyarrow` crate to dev/release/README.md (#7794)
alamb Jun 27, 2025
8d6cada
[Variant] Minor: make fields in `VariantDecimal*` private, add exampl…
alamb Jun 27, 2025
3183e03
Remove functions from parquet crate deprecated in or before 54.0.0 (#…
etseidl Jun 27, 2025
674dc17
Add Fixed, Uuid support to arrow-avro (#7557)
nathaniel-d-ef Jun 28, 2025
aa96097
Perf: Add prefix compare for inlined compare and change use of inline…
zhuqi-lucas Jun 29, 2025
c1a57cb
[Variant] Add negative tests for reading invalid primitive variant va…
superserious-dev Jun 29, 2025
19a14dc
Remove deprecated temporal functions (#7813)
etseidl Jun 29, 2025
a9f316b
Use in-memory buffer for arrow_writer benchmark (#7823)
jhorstmann Jun 29, 2025
bf6a97a
make builder public under experimental (#7825)
XiangpengHao Jun 30, 2025
6cd1949
Allow concating struct arrays with no fields (#7829)
AdamGS Jun 30, 2025
43f58b2
[Variant] Speedup `ObjectBuilder` (62x faster) (#7808)
friendlymatthew Jun 30, 2025
d0ef310
Update release instructions to not push tags until release is approve…
alamb Jul 1, 2025
959577d
[Variant] impl [Try]From for VariantDecimalXX types (#7809)
scovich Jul 1, 2025
af8564f
Minor: Update release schedule on README (#7838)
alamb Jul 1, 2025
2405798
[Variant] Add flag in `ObjectBuilder` to control validation behavior …
micoo227 Jul 1, 2025
248ee73
[VARIANT] Support both fallible and infallible access to variants (#7…
scovich Jul 1, 2025
6123956
Add Decimal type support to arrow-avro (#7832)
jecsand838 Jul 1, 2025
52ad7d7
Perf: Make sort string view fast(1.5X ~ 3X faster) (#7792)
zhuqi-lucas Jul 1, 2025
edd691b
Remove deprecated Arrow functions (#7830)
etseidl Jul 3, 2025
c3e3c03
[Variant] Follow up nits and uncomment test cases (#7846)
friendlymatthew Jul 3, 2025
a99f027
[Minor] Add Benchmark for RowConverter::append (#7853)
Dandandan Jul 3, 2025
91199c7
CSV error message has values transposed (#7851)
Omega359 Jul 3, 2025
81ab147
[VARIANT] Add support for the json_to_variant API (#7783)
harshmotw-db Jul 3, 2025
e6cb61f
Speedup sorting for inline views: 1.4x - 1.7x improvement (#7856)
Dandandan Jul 3, 2025
32caf76
Add features to configure flate2 (#7827)
zeevm Jul 3, 2025
53236b4
chore: update link for `row_filter.rs` (#7866)
haohuaijin Jul 4, 2025
f569f5d
Benchmark: Add rich testing cases for sort string(utf8) (#7867)
zhuqi-lucas Jul 4, 2025
5649e39
Improvements for parquet writing performance (25%-44%) (#7824)
jhorstmann Jul 4, 2025
13d79b3
[Variant] Make sure ObjectBuilder and ListBuilder to be finalized bef…
viirya Jul 4, 2025
54e4734
[Variant] Fuzz testing and benchmarks for vaildation (#7849)
carpecodeum Jul 5, 2025
6de3881
Fix RowConverter when FixedSizeList is not the last (#7789)
findepi Jul 5, 2025
a3584e5
Add Enum type support to arrow-avro and Minor Decimal type fix (#7852)
jecsand838 Jul 5, 2025
985ec7e
Add `get_ref/get_mut` to JSON Writer (#7854)
cetra3 Jul 5, 2025
57f96f2
Added number to string benches for json_writer (#7864)
abacef Jul 5, 2025
aef3bdd
[Variant] Support creating sorted dictionaries (#7833)
friendlymatthew Jul 5, 2025
df837a4
fix: Incorrect inlined string view comparison after Add prefix compar…
zhuqi-lucas Jul 7, 2025
58b897b
Move arrow-pyarrow tests that require `pyarrow` to be installed into …
alamb Jul 7, 2025
c26d154
[Variant] Remove dead code, add comments (#7861)
alamb Jul 7, 2025
213d3be
[Variant] List and object builders have no effect until finalized (#7…
scovich Jul 7, 2025
3126dad
[Variant] Remove superflous validate call and rename methods (#7871)
friendlymatthew Jul 7, 2025
38a7a1a
benchmark: Add StringViewArray gc benchmark with not null cases (#7877)
zhuqi-lucas Jul 8, 2025
6f33763
Fix union slice logical_nulls length (#7855)
codephage2020 Jul 8, 2025
9246872
[Variant] Introduce `parquet-variant-json` crate (#7862)
alamb Jul 8, 2025
ff3a2f2
Add tests for invalid variant metadata and value (#7885)
viirya Jul 10, 2025
abe8a04
docs: More docs to `BatchCoalescer` (#7891)
2010YOUY01 Jul 10, 2025
7595417
fix: `view_types` benchmark slice should follow by correct len array …
zhuqi-lucas Jul 10, 2025
55865d3
memory tracking with memory pool (#7303)
waynexia Jul 10, 2025
d7dae2c
[ARROW-RS-7820][Variant] Add tests for large variant lists (#7876)
klion26 Jul 10, 2025
4c088fe
Fix current CI failure (#7898)
viirya Jul 11, 2025
b2583b0
[Variant] Speedup validation (#7878)
friendlymatthew Jul 11, 2025
b16c540
Remove redundant is_err checks in Variant tests (#7897)
viirya Jul 11, 2025
b634638
Add arrow-avro support for bzip2 and xz compression (#7890)
jecsand838 Jul 11, 2025
7c42a83
[Variant] Define basic convenience methods for variant pathing (#7894)
scovich Jul 11, 2025
42b6c17
[Variant] Reduce variant-related struct sizes (#7888)
scovich Jul 11, 2025
387490a
fix: mark `DataType::Map` as unsupported in `RowConverter` (#7880)
rluvaton Jul 11, 2025
058243a
[Variant] Introduce parquet-variant-compute crate to transform batche…
harshmotw-db Jul 11, 2025
1b8cd9a
Update sysinfo requirement from 0.35.0 to 0.36.0 (#7904)
dependabot[bot] Jul 11, 2025
0269382
fix: Change panic to error in`take` kernel for StringArrary/BinaryArr…
chenkovsky Jul 11, 2025
ba751bf
Implement arrow-avro Reader and ReaderBuilder (#7834)
jecsand838 Jul 11, 2025
6d11232
add `garbage_collect_dictionary` to `arrow-select` (#7716)
davidhewitt Jul 11, 2025
7b219f9
perf: speed up StringViewArray gc 1.4 ~5.x faster (#7873)
zhuqi-lucas Jul 11, 2025
d8b5ef7
[Variant] Avoid superflous validation checks (#7906)
friendlymatthew Jul 12, 2025
fe77f2f
Add `VariantArray` and `VariantArrayBuilder` for constructing Arrow A…
alamb Jul 12, 2025
daf31be
[Variant] Add `VariantBuilder::new_with_buffers` to write to existing…
alamb Jul 13, 2025
d51fc45
feat: support `MapArray` in lexsort (#7882)
rluvaton Jul 14, 2025
5555d30
[Parquet] Use `u64` for `SerializedPageReaderState.offset` & `remaini…
JigaoLuo Jul 14, 2025
52fd59c
[Variant] Use simdutf8 for UTF-8 validation (#7908)
codephage2020 Jul 14, 2025
02e06c5
Add arrow-avro support for Duration type and minor fixes for UUID dec…
jecsand838 Jul 14, 2025
a653fd9
Restructure compare_greater function used in parquet statistics for b…
jhorstmann Jul 14, 2025
d534dd0
chore(dependabot): group tonic updates (#7925)
mbrobbel Jul 14, 2025
9c0cb9a
chore: bump MSRV to 1.84 (#7926)
mbrobbel Jul 15, 2025
b1a1864
Update bzip2 requirement from 0.4.4 to 0.6.0 (#7924)
mbrobbel Jul 15, 2025
c40830e
[Variant] test: add variant object tests with different sizes (#7896)
odysa Jul 15, 2025
7b7aad2
Upgrade tonic dependencies to 0.13.0 version (try 2) (#7839)
alamb Jul 16, 2025
0055f57
[Variant] Reserve capacity beforehand during large object building (#…
friendlymatthew Jul 16, 2025
7af62d5
[Variant] Support appending complex variants in `VariantBuilder` (#7914)
friendlymatthew Jul 16, 2025
d4c0a32
[Variant] Add `variant_get` compute kernel (#7919)
Samyak2 Jul 16, 2025
03a837e
Add tests for `BatchCoalescer::push_batch_with_filter`, fix bug (#7774)
alamb Jul 16, 2025
d809f19
[Variant] Add documentation, tests and cleaner api for Variant::get_p…
alamb Jul 17, 2025
7089786
[Variant] Avoid collecting offset iterator (#7934)
codephage2020 Jul 17, 2025
dfe907f
Minor: Support BinaryView and StringView builders in `make_builder` (…
kylebarron Jul 17, 2025
d0fa24e
[Variant] Impl `PartialEq` for VariantObject (#7943)
friendlymatthew Jul 17, 2025
233dad3
Optimize partition_validity function used in sort kernels (#7937)
jhorstmann Jul 18, 2025
722ef59
[Variant] Add ObjectBuilder::with_field for convenience (#7950)
alamb Jul 18, 2025
a984ca7
[Variant] Adding code to store metadata and value references in Varia…
abacef Jul 18, 2025
a5afda2
[Variant] VariantMetadata is allowed to contain the empty string (#7956)
scovich Jul 18, 2025
71dd48e
[Variant] Add `variant_kernels` benchmark (#7944)
alamb Jul 18, 2025
a15f345
[Variant] Add ListBuilder::with_value for convenience (#7959)
codephage2020 Jul 18, 2025
4f5ab12
[Test] Add tests for VariantList equality (#7953)
alamb Jul 18, 2025
55fbf5c
[Variant] remove VariantMetadata::dictionary_size (#7958)
codephage2020 Jul 18, 2025
99eb1bc
Add missing `parquet-variant-compute` crate to CI jobs (#7963)
alamb Jul 18, 2025
82821e5
arrow-ipc: Remove all abilities to preserve dict IDs (#7940)
brancz Jul 18, 2025
291e6e5
Add arrow-avro support for Impala Nullability (#7954)
veronica-m-ef Jul 21, 2025
b726b6f
Add additional integration tests to arrow-avro (#7974)
nathaniel-d-ef Jul 22, 2025
ed02131
arrow-schema: Remove dict_id from being required equal for merging (#…
brancz Jul 22, 2025
d4f1cfa
Implement Improved arrow-avro Reader Zero-Byte Record Handling (#7966)
jecsand838 Jul 22, 2025
6874ffa
[Variant] Avoid extra allocation in object builder (#7935)
klion26 Jul 22, 2025
dff67c9
GH-7686: [Parquet] Fix int96 min/max stats (#7687)
rahulketch Jul 22, 2025
f39461c
[Variant] Revisit VariantMetadata and Object equality (#7961)
friendlymatthew Jul 22, 2025
ec81db3
Add decimal32 and decimal64 support to Parquet, JSON and CSV readers …
CurtHagenlocher Jul 22, 2025
50f5562
Convert JSON to VariantArray without copying (8 - 32% faster) (#7911)
alamb Jul 23, 2025
a7f3ba8
Fix panic on lossy decimal to float casting: round to saturation for …
kosiew Jul 23, 2025
3e089d2
Perf: optimize actual_buffer_size to use only data buffer capacity fo…
zhuqi-lucas Jul 23, 2025
16794ab
Minor: Restore warning comment on Int96 statistics read (#7975)
alamb Jul 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
6 changes: 4 additions & 2 deletions .asf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,27 +16,29 @@
# under the License.

# Documentation can be found here:
# https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405038
# https://github.com/apache/infrastructure-asfyaml/blob/main/README.md

notifications:
commits: [email protected]
issues: [email protected]
pullrequests: [email protected]
discussions: [email protected]
jira_options: link label worklog
github:
description: "Official Rust implementation of Apache Arrow"
homepage: https://arrow.apache.org/
labels:
- arrow
- parquet
- object-store
- rust
enabled_merge_buttons:
squash: true
squash_commit_message: PR_TITLE_AND_DESC
merge: false
rebase: false
features:
issues: true
discussions: true
protected_branches:
main:
required_status_checks:
Expand Down
24 changes: 14 additions & 10 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -1,22 +1,26 @@
version: 2
updates:
- package-ecosystem: cargo
directory: "/"
schedule:
interval: daily
open-pull-requests-limit: 10
target-branch: main
labels: [ auto-dependencies, arrow ]
- package-ecosystem: cargo
directory: "/object_store"
directories:
- "/"
- "/arrow-pyarrow-integration-testing"
schedule:
interval: daily
open-pull-requests-limit: 10
target-branch: main
labels: [ auto-dependencies, object_store ]
labels: [auto-dependencies, arrow]
groups:
prost:
applies-to: version-updates
patterns:
- "prost*"
tonic:
applies-to: version-updates
patterns:
- "tonic*"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
open-pull-requests-limit: 10
labels: [ auto-dependencies ]
labels: [auto-dependencies]
25 changes: 11 additions & 14 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,28 @@
# Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax.

Closes #.
- Closes #NNN.

# Rationale for this change

<!--

Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->

# What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->

# Are there any user-facing changes?
# Are these changes tested?

We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?

# Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->

<!---
If there are any breaking changes to public APIs, please call them out.
-->
6 changes: 3 additions & 3 deletions .github/workflows/arrow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,11 +146,11 @@ jobs:
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
target: wasm32-unknown-unknown,wasm32-wasi
target: wasm32-unknown-unknown,wasm32-wasip1
- name: Build wasm32-unknown-unknown
run: cargo build -p arrow --no-default-features --features=json,csv,ipc,ffi --target wasm32-unknown-unknown
- name: Build wasm32-wasi
run: cargo build -p arrow --no-default-features --features=json,csv,ipc,ffi --target wasm32-wasi
- name: Build wasm32-wasip1
run: cargo build -p arrow --no-default-features --features=json,csv,ipc,ffi --target wasm32-wasip1

clippy:
name: Clippy
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/arrow_flight.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ jobs:
cargo test -p arrow-flight --all-features
- name: Test --examples
run: |
cargo test -p arrow-flight --features=flight-sql-experimental,tls --examples
cargo test -p arrow-flight --features=flight-sql,tls-ring --examples

vendor:
name: Verify Vendored Code
Expand Down
8 changes: 3 additions & 5 deletions .github/workflows/dev_pr/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,10 @@ arrow-flight:

parquet:
- changed-files:
- any-glob-to-any-file: [ 'parquet/**/*' ]
- any-glob-to-any-file:
- 'parquet/**/*'
- 'parquet-variant/**/*'

parquet-derive:
- changed-files:
- any-glob-to-any-file: [ 'parquet_derive/**/*' ]

object-store:
- changed-files:
- any-glob-to-any-file: [ 'object_store/**/*' ]
35 changes: 23 additions & 12 deletions .github/workflows/integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ on:
- arrow-avro/**
- arrow-ord/**
- arrow-pyarrow-integration-testing/**
- arrow-pyarrow/**
- arrow-schema/**
- arrow-select/**
- arrow-sort/**
Expand All @@ -61,12 +62,11 @@ jobs:
BUILD_DOCS_CPP: OFF
ARROW_INTEGRATION_CPP: ON
ARROW_INTEGRATION_CSHARP: ON
ARROW_INTEGRATION_GO: ON
ARROW_INTEGRATION_JAVA: ON
ARROW_INTEGRATION_JS: ON
ARCHERY_INTEGRATION_TARGET_IMPLEMENTATIONS: "rust"
# Disable nanoarrow integration, due to https://github.com/apache/arrow-rs/issues/5052
ARCHERY_INTEGRATION_WITH_NANOARROW: "0"
ARCHERY_INTEGRATION_WITH_GO: "1"
ARCHERY_INTEGRATION_WITH_JAVA: "1"
ARCHERY_INTEGRATION_WITH_JS: "1"
ARCHERY_INTEGRATION_WITH_NANOARROW: "1"
# https://github.com/apache/arrow/pull/38403/files#r1371281630
ARCHERY_INTEGRATION_WITH_RUST: "1"
# These are necessary because the github runner overrides $HOME
Expand Down Expand Up @@ -98,16 +98,26 @@ jobs:
with:
path: rust
fetch-depth: 0
- name: Checkout Arrow Go
uses: actions/checkout@v4
with:
repository: apache/arrow-go
path: go
- name: Checkout Arrow Java
uses: actions/checkout@v4
with:
repository: apache/arrow-java
path: java
- name: Checkout Arrow JavaScript
uses: actions/checkout@v4
with:
repository: apache/arrow-js
path: js
- name: Checkout Arrow nanoarrow
uses: actions/checkout@v4
with:
repository: apache/arrow-nanoarrow
path: nanoarrow
fetch-depth: 0
# Workaround https://github.com/rust-lang/rust/issues/125067
- name: Downgrade rust
working-directory: rust
run: rustup override set 1.77
- name: Build
run: conda run --no-capture-output ci/scripts/integration_arrow_build.sh $PWD /build
- name: Run
Expand Down Expand Up @@ -155,8 +165,9 @@ jobs:
- name: Run Rust tests
run: |
source venv/bin/activate
cargo test -p arrow --test pyarrow --features pyarrow
- name: Run tests
cd arrow-pyarrow-testing
cargo test
- name: Run Python tests
run: |
source venv/bin/activate
cd arrow-pyarrow-integration-testing
Expand Down
Loading
Loading