Skip to content

Conversation

AdamGS
Copy link
Contributor

@AdamGS AdamGS commented Sep 10, 2025

Which issue does this PR close?

Closes #17489.

Rationale for this change

Since Arrow 56 and #16690, support has been added to the two smaller decimal variants. This PR tries to bring support to them on-par with the support for the bigger types.

I've run into this issue when upgrading Vortex to use the upcoming release (50), one of our benchmarks fails with:

Error during planning: Execution error: Function 'sum' user-defined coercion failed with "Execution error: Sum not supported for Decimal32(7, 2)" No function matches the given name and argument types 'sum(Decimal32(7, 2))'. You might need to add explicit type casts.
	Candidate functions:
	sum(UserDefined)

What changes are included in this PR?

Adding support for handling decimal32/64 wherever other decimal types are handled. I've tried to use multiple search methods to find places where they matter, but there might be some matches/if conditions that make them hard to find.

I've also implemented support for AVG, SUM, FIRST_VALUE/LAST_VALUE and MIN/MAX. but I stopped pushing MEDIAN and MIN_MAX as I'm not sure if its desirable (at all) or as part of this PR. Seems like implementing these aggregations is required to get the test suite to ✅, so I'm working on adding everything.

Are these changes tested?

The testing strategy to some of these parts is unclear to me, both around SQL but also for aggregate functions like SUM and AVG.

Are there any user-facing changes?

Code that used to fail while reading some decimals should now work.

@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions common Related to common crate proto Related to proto crate functions Changes to functions implementation labels Sep 10, 2025
@AdamGS AdamGS force-pushed the adamg/extended-decimal-support branch 3 times, most recently from e8cf5e2 to 4bad6d6 Compare September 10, 2025 10:18
@github-actions github-actions bot added the core Core DataFusion crate label Sep 10, 2025
@AdamGS AdamGS force-pushed the adamg/extended-decimal-support branch from 4bad6d6 to 8e6338c Compare September 10, 2025 10:32
@AdamGS AdamGS force-pushed the adamg/extended-decimal-support branch from d2e7917 to 3bba269 Compare September 10, 2025 12:40
@AdamGS AdamGS marked this pull request as ready for review September 10, 2025 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
common Related to common crate core Core DataFusion crate functions Changes to functions implementation logical-expr Logical plan and expressions proto Related to proto crate sql SQL Planner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Decimal32/64 aren't as well supported as as the 128 and 256 bit variants
1 participant