Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
351 commits
Select commit Hold shift + click to select a range
d818186
add option to control geoparquet version
Maxxen Oct 3, 2025
c558afa
actually, only write gpq1 version 1 unless opt-in
Maxxen Oct 3, 2025
e61a4e2
Switch to separate Query() method to get a query
Mytherin Oct 3, 2025
33b638b
Bump DuckLake to latest of V1.4 (#19237)
Mytherin Oct 3, 2025
3f51c67
Bump mysql and sqlite (#19240)
Mytherin Oct 3, 2025
dabebf3
Don't write parquet-native `GEOMETRY` by default, add option to contr…
Mytherin Oct 3, 2025
7aa6348
add support for writing geoparquet with v2 metadata too
Maxxen Oct 3, 2025
2549afa
When executing a relation, generate a query to set if it is not a que…
Mytherin Oct 3, 2025
b75f735
rework ParquetColumnSchema constructors
Tishj Oct 3, 2025
4476118
bump extensions
samansmink Oct 3, 2025
db3862c
set the field id for primitive columns (this logic was lost in a refa…
Tishj Oct 3, 2025
419111f
bump extensions
samansmink Oct 3, 2025
8a74cb9
add support for writing geoparquet with v2 metadata too (#19246)
Mytherin Oct 3, 2025
253ebab
Bump: iceberg (#19250)
Mytherin Oct 3, 2025
b390a7c
Bump: avro, httpfs (#19248)
Mytherin Oct 3, 2025
a99f5be
feat: Fix length of sort keys
krlmlr Oct 4, 2025
5a40d9e
Don't create recursive choice matchers
Dtenwolde Oct 6, 2025
3f0f5bc
the ColumnWriter now owns the ParquetColumnSchema, removed the FillPa…
Tishj Oct 6, 2025
53030dc
missing assertion
Tishj Oct 6, 2025
a2281ac
Create script that generates the method and definitions
Dtenwolde Oct 6, 2025
aee2e0e
Include proper transformer
Dtenwolde Oct 6, 2025
e7558e7
Include proper transformer again
Dtenwolde Oct 6, 2025
faeab11
Add new Transform overload that doesn't require casting
Dtenwolde Oct 6, 2025
066f364
Clean up child
Dtenwolde Oct 6, 2025
6dffa1f
Format fix
Dtenwolde Oct 6, 2025
377815a
Include sqlstatement
Dtenwolde Oct 6, 2025
1cfef17
bump duckdb-azure ref to HEAD for 1.4.1
benfleis Oct 6, 2025
6b51820
Catch NotImplementedExceptions
Dtenwolde Oct 6, 2025
467b17a
Set the statament location and length
Dtenwolde Oct 6, 2025
f908102
Fix typo in transformer_stub_generator.py, remove set_statement
Dtenwolde Oct 6, 2025
313ad8b
initial implementation for determining shredding from analysis of fir…
Tishj Oct 6, 2025
11b9166
Dont build set statement
Dtenwolde Oct 6, 2025
f7793a2
finally create the new child 'typed_value' column writer for VARIANT …
Tishj Oct 6, 2025
e49deab
Remove print. Add error message to parser
Dtenwolde Oct 6, 2025
b62afab
Format fix
Dtenwolde Oct 6, 2025
bd731e9
bring back the 'preprocessing' logic, since we now need to first anal…
Tishj Oct 6, 2025
25f63eb
make explicit shredding work again
Tishj Oct 6, 2025
cdf9e7c
bump duckdb-azure ref for 1.4.1 (#19275)
Mytherin Oct 6, 2025
65c3c88
fix some problems, highlighted a new problem: the transformed buffer …
Tishj Oct 7, 2025
546ac7b
Add configuration to test
Dtenwolde Oct 7, 2025
412202f
Dont do optionally quoted as already handled in parser
Dtenwolde Oct 7, 2025
7005a01
Use ErrorData rather than raw string
Dtenwolde Oct 7, 2025
2b5e270
make sure preprocessing can happen everywhere, which requires the Par…
Tishj Oct 7, 2025
2dfea98
Format fix
Dtenwolde Oct 7, 2025
f0349c9
Fix up loadable extension demo
Dtenwolde Oct 7, 2025
9518107
Add testing peg parser config to workflow
Dtenwolde Oct 7, 2025
81f35db
Change name
Dtenwolde Oct 7, 2025
1d32572
fix map child schema creation
Tishj Oct 7, 2025
e43ddc1
Merge branch 'main' into InternalException_reference
maiadegraaf Oct 7, 2025
290bde6
comments
Tishj Oct 7, 2025
b1123dd
add Geometry::MAX_RECURSION_DEPTH
maiadegraaf Oct 7, 2025
8037d7e
Exclude loadable_parser_override.test
Dtenwolde Oct 7, 2025
a884bef
Fix regex optimization to remove flags when converting to contains
mlafeldt Oct 7, 2025
a695b89
Merge branch 'write_parquet_variants_shredding' into write_parquet_va…
Tishj Oct 7, 2025
5511d3c
remove outdated fixme
Tishj Oct 7, 2025
c7b116b
Add test
mlafeldt Oct 7, 2025
12f9c34
add back the field id to the list schema
Tishj Oct 7, 2025
1ab986a
add back the field id to the map schema
Tishj Oct 7, 2025
177e447
fix up the 'RETURN STATS' path, using the ChildWriters instead of the…
Tishj Oct 7, 2025
2b78eee
add missing header
Tishj Oct 7, 2025
7d019a5
Fix regex optimization to remove flags when converting to contains (#…
Mytherin Oct 7, 2025
24637e3
Merge branch 'main' into hjiang/provide-force-assert-macrp
dentiny Oct 8, 2025
ed88b0e
Remove BoundCTENode
Mytherin Oct 8, 2025
0b33496
feat: Fix length of sort keys (#19260)
hannes Oct 8, 2025
10f8766
Pass lambda bindings to next binder (#19144)
hannes Oct 8, 2025
f987978
use cross-compilation for static libs on osx, this avoids relying on …
hannes Oct 8, 2025
15f3662
bump Julia to v1.4.1
c-herrewijn Oct 8, 2025
047c4c2
Merge remote-tracking branch 'upstream/main' into write_parquet_varia…
Tishj Oct 8, 2025
c94b23d
Merge remote-tracking branch 'upstream' into parse_result
Dtenwolde Oct 8, 2025
6360e09
Merge remote-tracking branch 'upstream' into use_statement
Dtenwolde Oct 8, 2025
4e6a45d
Merge remote-tracking branch 'upstream' into issues/5955
Dtenwolde Oct 8, 2025
ac2d28e
remove race on '.empty()' check
Tishj Oct 8, 2025
fb79ee2
bump Julia to v1.4.1 (#19306)
Mytherin Oct 8, 2025
e7cf8f2
wrong flag
hannes Oct 8, 2025
e46e48a
remove old sort code and everything associated with it
Oct 8, 2025
716e174
Use cross-compilation for static libs on OSX (#19304)
hannes Oct 8, 2025
e70ce1a
fix httpfs patch
maiadegraaf Oct 8, 2025
d02d2a6
Internal #6190: AsOf Source Cleanup
hawkfish Oct 8, 2025
68953e5
Simplify logic, remove unneeded cast
Dtenwolde Oct 9, 2025
96541c4
hugeint_t: Avoid implicit conversions to uhugeint_t
carlopi Oct 9, 2025
a4ec500
hugeint_t: Avoid copies around comparisons
carlopi Oct 9, 2025
dcae1e2
hugeint_t: Remove specialized code for hugeint comparison
carlopi Oct 9, 2025
f94ad63
Patch inet (bottom of the range is wrong, but...)
carlopi Oct 9, 2025
4b21904
Allow multiple read-only attaches to the same database across databas…
Mytherin Oct 9, 2025
fc40e7c
Add missing include
Mytherin Oct 9, 2025
5d9469e
Remove BoundRecursiveCTENode
Mytherin Oct 9, 2025
2ddc243
Remove bound query node casts
Mytherin Oct 9, 2025
3b5188f
Remove unused type
Mytherin Oct 9, 2025
b62e56d
GetSelectList is only used for a verifier - move it to that verifier
Mytherin Oct 9, 2025
f9ca8ba
Remove unused file
Mytherin Oct 9, 2025
f91f8d9
Move stored_cte_map to referenes
Mytherin Oct 9, 2025
1a77d4b
implement comparison logic for Variants
Tishj Oct 9, 2025
ead6ebe
add initial tests
Tishj Oct 9, 2025
42f22f0
Remove outdated file
carlopi Oct 9, 2025
8b65c04
add parquet filter pushdown test
Tishj Oct 9, 2025
a60c63c
steal the list_distinct test for variants
Tishj Oct 9, 2025
77d4948
also steal the struct distinct tests
Tishj Oct 9, 2025
f019fde
Merge remote-tracking branch 'upstream/main' into write_parquet_varia…
Tishj Oct 9, 2025
cac7f17
Merge branch 'write_parquet_variants_shredding' into write_parquet_va…
Tishj Oct 9, 2025
94a7b03
add new metrics
maiadegraaf Oct 9, 2025
0ffbf15
Allow multiple read-only attaches to the same database across databas…
Mytherin Oct 9, 2025
dcbdc89
Merge remote-tracking branch 'upstream/v1.4-andium' into merge_andium…
Tishj Oct 9, 2025
bfc6eba
WIP: remove CTENode
Mytherin Oct 9, 2025
17bf45b
Binder: BoundQueryNode cleanup (#19322)
Mytherin Oct 9, 2025
192fbda
Remove old sort code (#19321)
Mytherin Oct 9, 2025
46beeea
hugeint_t fixes (#19318)
Mytherin Oct 9, 2025
ff17e69
Merge remote-tracking branch 'upstream/main' into merge_andium_into_main
Tishj Oct 9, 2025
40243d0
Merge remote-tracking branch 'upstream/v1.4-andium' into merge_andium…
Tishj Oct 9, 2025
d223c6e
Internal #6190: AsOf Threading
hawkfish Oct 9, 2025
62dbe52
Merge branch 'main' into asof-source
hawkfish Oct 9, 2025
a6238fb
Fix attach issue in read_duckdb
Mytherin Oct 10, 2025
765d9bb
remove test
dentiny Oct 10, 2025
b83205f
[Parquet] Implement shredded `VARIANT` support for COPY TO parquet (#…
Mytherin Oct 10, 2025
5d4ca55
Merge remote-tracking branch 'upstream/main' into merge_andium_into_main
Tishj Oct 10, 2025
6676055
expose duckdb_table_description_get_column_count and duckdb_table_des…
taniabogatsch Oct 10, 2025
17a8859
Merge remote-tracking branch 'upstream/main' into write_parquet_varia…
Tishj Oct 10, 2025
a3693ce
Add node handlers for base_leaf
artjomPlaunov Oct 6, 2025
a44547d
Fix scoping for node handlers
artjomPlaunov Oct 9, 2025
088b14f
remove unused variable
Tishj Oct 10, 2025
c347029
reuse the local state's transform data
Tishj Oct 10, 2025
e4e42f9
undo cosmetic change to test
Tishj Oct 10, 2025
e5189e4
Merge remote-tracking branch 'upstream/main' into variant_comparison_…
Tishj Oct 10, 2025
ab40d09
Use correct binder for Finalize bind
Mytherin Oct 10, 2025
852d439
Fix for CTE node deserialization
Mytherin Oct 10, 2025
c6c0e9e
Partially move set operation binding to be more regular
Mytherin Oct 10, 2025
5bfa28e
Avoid peeking into BoundSelectNode in setop bind
Mytherin Oct 10, 2025
31c4ede
add DUCKDB_DATA_DIR
hmeriann Oct 10, 2025
5714702
Clean-up: remove internal select node bind code
Mytherin Oct 10, 2025
4847a70
init
d-justen Oct 10, 2025
a89118a
relocate the python sqllogic code to the dedicated repository
Tishj Oct 10, 2025
564a769
undo a change that looked innocuous that somehow caused regressions
Tishj Oct 10, 2025
a6959a6
while we're at it, might as well remove the pointer dereferencing fro…
Tishj Oct 10, 2025
b301e4a
fix null order
d-justen Oct 10, 2025
9525f84
WIP: move set operations to regular binding
Mytherin Oct 10, 2025
12c7aed
Immediately plan union by name re-ordering
Mytherin Oct 10, 2025
fc2da14
Merge andium into main (#19325)
Mytherin Oct 10, 2025
03f18f0
Remove special BoundSetOpChild
Mytherin Oct 10, 2025
30da910
Merge branch 'setopbinding' into remotectenode
Mytherin Oct 10, 2025
1a4d2f9
Add missing include
Mytherin Oct 10, 2025
b2fbb30
Merge branch 'setopbinding' into remotectenode
Mytherin Oct 10, 2025
16370e6
Merge remote-tracking branch 'upstream/main' into write_parquet_varia…
Tishj Oct 10, 2025
bbe6551
Add missing includes
Mytherin Oct 10, 2025
12ffa73
[C API] Expose column count and type for table description (#19334)
Mytherin Oct 10, 2025
95d43bd
Internal #6190: IEJoin Threading
hawkfish Oct 10, 2025
db3c51b
Internal #6190: IEJoin Threading
hawkfish Oct 10, 2025
9d77bcf
Relocate the python sqllogic code to the dedicated repository (#19340)
Mytherin Oct 10, 2025
f0b6367
Add backwards compatibility test for reading CTEs
Mytherin Oct 10, 2025
cba7992
Remove unnecessary add context
Mytherin Oct 10, 2025
e128429
Add test for v1.4
Mytherin Oct 10, 2025
a6dd442
improve docs and return error, if size > capacity
taniabogatsch Oct 11, 2025
39675e4
remove size check again
taniabogatsch Oct 11, 2025
0e5bf48
Binder: Rework the way set operations work so that we can use the sta…
Mytherin Oct 11, 2025
f971bc2
Make Binding members private and move to using getters in the binding…
Mytherin Oct 11, 2025
2aa3d23
Add require
Mytherin Oct 11, 2025
e2665de
Debug mode fixes
Mytherin Oct 11, 2025
7fc373a
Avoid keeping shared ptr to ClientContext in QueryContext
Mytherin Oct 11, 2025
cb636bc
Rework CTE binding: remove CTENode, and bind CommonTableExpressionMap…
Mytherin Oct 11, 2025
2762f1a
Avoid keeping shared ptr to ClientContext in QueryContext (#19353)
Mytherin Oct 11, 2025
54bcb3a
Merge branch 'main' into lazyctebind
Mytherin Oct 11, 2025
9131ba1
Internal #6190: AsOf Threading (#19328)
Mytherin Oct 13, 2025
f7945f6
[Indexes] Buffer Managed Indexes Part 4: Segment Handles for Base Lea…
Mytherin Oct 13, 2025
029a5b8
Provide force assertion macros (#19212)
Mytherin Oct 13, 2025
3f06467
update docs
taniabogatsch Oct 13, 2025
7a442cd
Rework CTEs to support schemas, and use actual schema name "recurring…
Mytherin Oct 13, 2025
c56090f
make work
d-justen Oct 13, 2025
1af7f2b
Merge branch 'main' into arg_max_null_val
d-justen Oct 13, 2025
fa923eb
I stand corrected again, this can not use the global state, because t…
Tishj Oct 13, 2025
40bfcf4
Implement circular CTE reference detection using the cte bindings in …
Mytherin Oct 13, 2025
5249835
move parquet filter test to a different PR
Tishj Oct 13, 2025
788ecf8
Move CTEBinding members to private
Mytherin Oct 13, 2025
5f9e410
add to optimizer rule
d-justen Oct 13, 2025
4a221c6
Parse result for PEG transformer (#19229)
Mytherin Oct 13, 2025
7bc5d9e
Allow * NOT SIMILAR TO 'pattern' (#19232)
Mytherin Oct 13, 2025
9b8604e
[C API] Improve documentation on list vectors (#19349)
hannes Oct 13, 2025
45877d8
Lazy CTE binding
Mytherin Oct 13, 2025
1a638fa
fix
d-justen Oct 13, 2025
3166d30
review feedback and refactor BaseProfiler
taniabogatsch Oct 13, 2025
fe1079d
move test/sql/copy/parquet/parquet_encrypted_tpch_httpfs.test_slow an…
hmeriann Oct 13, 2025
34b2344
remove unuset env added before
hmeriann Oct 13, 2025
38b4212
fix
d-justen Oct 13, 2025
12c4134
update use_instead
taniabogatsch Oct 13, 2025
99bc8dc
Add lazy CTE bind test
Mytherin Oct 13, 2025
604e5ce
Make Binding CTEs lazy, and other CTE binding refactors (#19372)
Mytherin Oct 13, 2025
bc61f69
Internal #6210: IEJoin Threading (#19361)
Mytherin Oct 13, 2025
9c8d4db
[CI] Move two test cases from duckdb-httpfs repo (#19368)
Mytherin Oct 13, 2025
7621d1b
[C API] Update misleading "use instead" (#19371)
Mytherin Oct 13, 2025
f55ed16
enable -Wtype-limits warning
hannes Oct 14, 2025
6ad4697
fix httpfs patch
maiadegraaf Oct 14, 2025
3ee7f20
remove special variant case
Tishj Oct 14, 2025
3323e55
fix test generation
taniabogatsch Oct 14, 2025
caeeb1a
update enum util
taniabogatsch Oct 14, 2025
5ab6ecc
add appender profiling test
maiadegraaf Oct 14, 2025
760afef
nits
taniabogatsch Oct 14, 2025
993dbfa
Update issue template
szarnyasg Oct 14, 2025
19acd1e
add a VariantVisitor class, to deduplicate the variant traversal logic
Tishj Oct 14, 2025
d569f2d
use the VariantVisitor for the cast to JSON as well
Tishj Oct 14, 2025
5294da2
Enable -Wtype-limits warning and fix warnings (#19382)
Mytherin Oct 14, 2025
ffff8f8
fix compilation issues
Tishj Oct 14, 2025
baafa66
Switch RowVersionManager to using the FixedSizeAllocator
Mytherin Oct 14, 2025
14238f5
de-template
d-justen Oct 14, 2025
c2c5f12
Clean-up: move accessors to functions
Mytherin Oct 14, 2025
d23c4c9
Merge remote-tracking branch 'upstream' into use_statement
Dtenwolde Oct 14, 2025
a081968
Remove some comments
Dtenwolde Oct 14, 2025
a832f4f
Remove the stub generator. Going to rework that later
Dtenwolde Oct 14, 2025
4c573db
Use IndexPointer metadata to store whether or not they are set
Mytherin Oct 14, 2025
7483bce
Move constant id behind getter
Mytherin Oct 14, 2025
18921b0
add test for non-query appender
maiadegraaf Oct 14, 2025
f42b64a
Merge branch 'main' into InternalException_reference
maiadegraaf Oct 14, 2025
a4cf987
not building imdb all the time
hannes Oct 14, 2025
b020537
tidy
d-justen Oct 14, 2025
3a20f28
fix patches
maiadegraaf Oct 14, 2025
2df886e
also make use of the visitors in 'variant_to_variant'
Tishj Oct 14, 2025
567581f
override instead of final for clang-tidy
Mytherin Oct 14, 2025
124c683
CSV Reader: make line reconstruction deal with buffer_pos == buffer_size
Mytherin Oct 14, 2025
a1c8c97
cleanup
d-justen Oct 14, 2025
b65b2d9
[Profiling] Add metrics for ATTACH / CHECKPOINT / WAL replay (#19367)
Mytherin Oct 14, 2025
5e73b5c
more compilation issues
Tishj Oct 14, 2025
5c834d8
copy of variant_typeof binddata fix
Tishj Oct 14, 2025
3df33d9
patches
maiadegraaf Oct 14, 2025
0d3d29f
tidy
d-justen Oct 14, 2025
ef047f7
Update issue template (#19384)
Mytherin Oct 14, 2025
d25db2f
Correctly tag memory allocated in the RowVersionManager with BASE_TABLE
Mytherin Oct 14, 2025
746404c
Not building imdb all the time (#19388)
Mytherin Oct 14, 2025
b1232fc
Currently, all functions that access or modify the `UpdateInfo` chain…
Captain32 Oct 14, 2025
869c613
CSV Reader: make line reconstruction deal with buffer_pos == buffer_s…
Mytherin Oct 15, 2025
4fdf341
[Dev] `variant_extract` BindData copy fix (#19394)
Mytherin Oct 15, 2025
7d2aae0
Merge v1.4-andium into main
Mytherin Oct 15, 2025
8704368
[Dev] Add a `VARIANT` visitor class (#19387)
Mytherin Oct 15, 2025
a110659
Switch RowVersionManager to using the FixedSizeAllocator (#19392)
hannes Oct 15, 2025
0babf65
[Profiling][CAPI] Add appender profiling test (#19383)
hannes Oct 15, 2025
cf51328
Use statement for PEG Parser transformer (#19302)
hannes Oct 15, 2025
f008053
final fix
maiadegraaf Oct 15, 2025
f1b2934
Merge remote-tracking branch 'upstream/main' into variant_comparison_…
Tishj Oct 15, 2025
a8303e6
`old_transactions` is no longer necessary (#19381)
Mytherin Oct 15, 2025
71ab357
Move broken files to broken directory
Mytherin Oct 15, 2025
7977b60
Add Null Support for TopNWindowElimination Rule (#19370)
Mytherin Oct 15, 2025
438a121
first commit for 'variant_normalize'
Tishj Oct 15, 2025
02512cf
further tinkering, added an optional visitor callback: 'VisitMetadata'
Tishj Oct 15, 2025
42d4248
fix a bug shadowing variables, seems to function correctly now
Tishj Oct 15, 2025
96f6859
remove some bloat, there was no need to have a 'ToVariantGlobalResult…
Tishj Oct 15, 2025
33f6ef9
some renaming, the struct outgrew its original purpose
Tishj Oct 15, 2025
6e197e9
Use const references in exceptions varargs (#18723)
Mytherin Oct 15, 2025
6f71834
comments
Tishj Oct 15, 2025
ea59338
forgot to slice the keys entry
Tishj Oct 15, 2025
7aed30e
push a collation for variant, which runs the variant through 'variant…
Tishj Oct 15, 2025
2cf2a67
add test with extracted variant, to 'variant_distinct.test'
Tishj Oct 15, 2025
5d422e4
Merge v1.4-andium into main (#19402)
Mytherin Oct 15, 2025
8afbd2e
fix tests
Tishj Oct 15, 2025
e36fc5b
[Variant] Implement comparison operation logic for Variant columns (#…
Mytherin Oct 16, 2025
2bf2967
Merge remote-tracking branch 'upstream/main' into write_parquet_varia…
Tishj Oct 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
57 changes: 27 additions & 30 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,38 +8,35 @@ body:
value: >
DuckDB has several repositories for different components, please make sure you're raising your issue in the correct one:

* [Documentation/website](https://github.com/duckdb/duckdb-web/issues/new)
* APIs:
* [duckdb-java](https://github.com/duckdb/duckdb-java/issues/new)
* [duckdb-node](https://github.com/duckdb/duckdb-node/issues/new)
* [duckdb-node-neo](https://github.com/duckdb/duckdb-node-neo/issues/new)
* [duckdb-odbc](https://github.com/duckdb/duckdb-odbc/issues/new)
* [duckdb-python](https://github.com/duckdb/duckdb-python/issues/new)
* [duckdb-r](https://github.com/duckdb/duckdb-r/issues/new)
* [duckdb-rs](https://github.com/duckdb/duckdb-rs/issues/new)
* [duckdb-swift](https://github.com/duckdb/duckdb-swift/issues/new)
* [duckdb-wasm](https://github.com/duckdb/duckdb-wasm/issues/new)
* [go-duckdb](https://github.com/marcboeker/go-duckdb/issues/new)
* [Documentation](https://github.com/duckdb/duckdb-web/issues/new)
* Clients:
[Go](https://github.com/duckdb/duckdb-go/issues/new),
[Java (JDBC)](https://github.com/duckdb/duckdb-java/issues/new),
[Node.js](https://github.com/duckdb/duckdb-node-neo/issues/new),
[ODBC](https://github.com/duckdb/duckdb-odbc/issues/new),
[Python](https://github.com/duckdb/duckdb-python/issues/new),
[R](https://github.com/duckdb/duckdb-r/issues/new),
[Rust](https://github.com/duckdb/duckdb-rs/issues/new),
[WebAssembly (Wasm)](https://github.com/duckdb/duckdb-wasm/issues/new)
* Extensions:
* [AWS extension](https://github.com/duckdb/duckdb-aws/issues/new)
* [Azure extension](https://github.com/duckdb/duckdb-azure/issues/new)
* [Delta extension](https://github.com/duckdb/duckdb-delta/issues/new)
* [Encodings extension](https://github.com/duckdb/duckdb-encodings/issues/new)
* [Excel extension](https://github.com/duckdb/duckdb-excel/issues/new)
* [fts (full text search) extension](https://github.com/duckdb/duckdb-fts/issues/new)
* [httpfs extension](https://github.com/duckdb/duckdb-httpfs/issues/new)
* [Iceberg extension](https://github.com/duckdb/duckdb-iceberg/issues/new)
* [inet extension](https://github.com/duckdb/duckdb-inet/issues/new)
* [MySQL extension](https://github.com/duckdb/duckdb-mysql/issues/new)
* [Postgres scanner](https://github.com/duckdb/duckdb-postgres/issues/new)
* [Spatial extension](https://github.com/duckdb/duckdb-spatial/issues/new)
* [SQLite scanner](https://github.com/duckdb/duckdb-sqlite/issues/new)
* [UI](https://github.com/duckdb/duckdb-ui/issues/new)
* [VSS extension](https://github.com/duckdb/duckdb-vss/issues/new)
* Connectors:
* [dbt-duckdb](https://github.com/duckdb/dbt-duckdb)
[`aws`](https://github.com/duckdb/duckdb-aws/issues/new),
[`azure`](https://github.com/duckdb/duckdb-azure/issues/new),
[`delta`](https://github.com/duckdb/duckdb-delta/issues/new),
[`ducklake`](https://github.com/duckdb/duckdb-ducklake/issues/new),
[`encodings`](https://github.com/duckdb/duckdb-encodings/issues/new),
[`excel`](https://github.com/duckdb/duckdb-excel/issues/new),
[`fts`](https://github.com/duckdb/duckdb-fts/issues/new),
[`httpfs`](https://github.com/duckdb/duckdb-httpfs/issues/new),
[`iceberg`](https://github.com/duckdb/duckdb-iceberg/issues/new),
[`inet`](https://github.com/duckdb/duckdb-inet/issues/new),
[`mysql`](https://github.com/duckdb/duckdb-mysql/issues/new),
[`postgres`](https://github.com/duckdb/duckdb-postgres/issues/new),
[`spatial`](https://github.com/duckdb/duckdb-spatial/issues/new),
[`sqlite`](https://github.com/duckdb/duckdb-sqlite/issues/new),
[`ui`](https://github.com/duckdb/duckdb-ui/issues/new),
[`vss`](https://github.com/duckdb/duckdb-vss/issues/new)

If none of the above repositories are applicable, feel free to raise it in this one
If the issue occurs in core DuckDB (e.g., a SQL query crashes or returns incorrect results) or if the issue is in the DuckDB command line client, feel free to raise it in this repository.

Please report security vulnerabilities using GitHub's [report vulnerability form](https://github.com/duckdb/duckdb/security/advisories/new).

Expand Down
3 changes: 2 additions & 1 deletion .github/config/extensions/avro.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ if (NOT MINGW)
duckdb_extension_load(avro
LOAD_TESTS DONT_LINK
GIT_URL https://github.com/duckdb/duckdb-avro
GIT_TAG 0c97a61781f63f8c5444cf3e0c6881ecbaa9fe13
GIT_TAG 7b75062f6345d11c5342c09216a75c57342c2e82
APPLY_PATCHES
)
endif()
2 changes: 1 addition & 1 deletion .github/config/extensions/aws.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ if (NOT MINGW AND NOT ${WASM_ENABLED})
duckdb_extension_load(aws
### TODO: re-enable LOAD_TESTS
GIT_URL https://github.com/duckdb/duckdb-aws
GIT_TAG 812ce80fde0bfa6e4641b6fd798087349a610795
GIT_TAG 18803d5e55b9f9f6dda5047d0fdb4f4238b6801d
)
endif()
2 changes: 1 addition & 1 deletion .github/config/extensions/azure.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ if (NOT MINGW AND NOT ${WASM_ENABLED})
duckdb_extension_load(azure
LOAD_TESTS
GIT_URL https://github.com/duckdb/duckdb-azure
GIT_TAG 5e458fcc466d2bc421922b11f4316564e3017800
GIT_TAG 0709c0fa1cf67a668b58b1f06ff3e5fc1696e10a
)
endif()
2 changes: 1 addition & 1 deletion .github/config/extensions/ducklake.cmake
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
duckdb_extension_load(ducklake
DONT_LINK
GIT_URL https://github.com/duckdb/ducklake
GIT_TAG 45788f0a875844ac8fed048c99b87f7f4b1c2ac1
GIT_TAG f134ad86f2f6e7cdf4133086c38ecd9c48f1a772
)
4 changes: 2 additions & 2 deletions .github/config/extensions/httpfs.cmake
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
duckdb_extension_load(httpfs
LOAD_TESTS
GIT_URL https://github.com/duckdb/duckdb-httpfs
GIT_TAG 0518838dae609ab8e8ae66960ce982b839754075
INCLUDE_DIR src/include
GIT_TAG 8356a9017444f54018159718c8017ff7db4ea756
APPLY_PATCHES
INCLUDE_DIR src/include
)
1 change: 0 additions & 1 deletion .github/config/extensions/iceberg.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ IF (NOT WIN32)
else ()
set(LOAD_ICEBERG_TESTS "")
endif()

if (NOT MINGW AND NOT ${WASM_ENABLED})
duckdb_extension_load(iceberg
# ${LOAD_ICEBERG_TESTS} TODO: re-enable once autoloading test is fixed
Expand Down
1 change: 1 addition & 0 deletions .github/config/extensions/inet.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@ duckdb_extension_load(inet
GIT_TAG f6a2a14f061d2dfccdb4283800b55fef3fcbb128
INCLUDE_DIR src/include
TEST_DIR test/sql
APPLY_PATCHES
)
2 changes: 1 addition & 1 deletion .github/config/extensions/mysql_scanner.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ if (NOT MINGW AND NOT ${WASM_ENABLED} AND NOT ${MUSL_ENABLED})
DONT_LINK
LOAD_TESTS
GIT_URL https://github.com/duckdb/duckdb-mysql
GIT_TAG 8a32d4e069438585e80494e296e407653aebfed3
GIT_TAG c80647b33972c150f0bd0001c35085cefdc82d1e
)
endif()
1 change: 1 addition & 0 deletions .github/config/extensions/spatial.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@ duckdb_extension_load(spatial
GIT_TAG a6a607fe3a98ef9ad4bed218490b770f725fbc12
INCLUDE_DIR src/spatial
TEST_DIR test/sql
APPLY_PATCHES
)
endif()
2 changes: 1 addition & 1 deletion .github/config/extensions/sqlite_scanner.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ endif()
duckdb_extension_load(sqlite_scanner
${STATIC_LINK_SQLITE} LOAD_TESTS
GIT_URL https://github.com/duckdb/duckdb-sqlite
GIT_TAG 833e105cbcaa0f6e8d34d334f3b920ce86f6fdf9
GIT_TAG 0c93d610af1e1f66292559fcf0f01a93597a98b6
)
13 changes: 13 additions & 0 deletions .github/patches/extensions/avro/fix.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
diff --git a/src/field_ids.cpp b/src/field_ids.cpp
index d197f8d..52fb48c 100644
--- a/src/field_ids.cpp
+++ b/src/field_ids.cpp
@@ -5,6 +5,8 @@ namespace duckdb {

namespace avro {

+constexpr const char *FieldID::DUCKDB_FIELD_ID;
+
FieldID::FieldID() : set(false) {
}

181 changes: 31 additions & 150 deletions .github/patches/extensions/httpfs/fix.patch
Original file line number Diff line number Diff line change
@@ -1,157 +1,38 @@
diff --git a/src/httpfs.cpp b/src/httpfs.cpp
index 802581e..a11af95 100644
--- a/src/httpfs.cpp
+++ b/src/httpfs.cpp
@@ -729,7 +729,7 @@ void HTTPFileHandle::LoadFileInfo() {
return;
} else {
// HEAD request fail, use Range request for another try (read only one byte)
- if (flags.OpenForReading() && res->status != HTTPStatusCode::NotFound_404) {
+ if (flags.OpenForReading() && res->status != HTTPStatusCode::NotFound_404 && res->status != HTTPStatusCode::MovedPermanently_301) {
auto range_res = hfs.GetRangeRequest(*this, path, {}, 0, nullptr, 2);
if (range_res->status != HTTPStatusCode::PartialContent_206 &&
range_res->status != HTTPStatusCode::Accepted_202 && range_res->status != HTTPStatusCode::OK_200) {
diff --git a/src/httpfs_extension.cpp b/src/httpfs_extension.cpp
index 79d9923..9070621 100644
--- a/src/httpfs_extension.cpp
+++ b/src/httpfs_extension.cpp
@@ -70,7 +70,7 @@ static void LoadInternal(ExtensionLoader &loader) {
config.AddExtensionOption("ca_cert_file", "Path to a custom certificate file for self-signed certificates.",
LogicalType::VARCHAR, Value(""));
// Global S3 config
- config.AddExtensionOption("s3_region", "S3 Region", LogicalType::VARCHAR, Value("us-east-1"));
+ config.AddExtensionOption("s3_region", "S3 Region", LogicalType::VARCHAR);
config.AddExtensionOption("s3_access_key_id", "S3 Access Key ID", LogicalType::VARCHAR);
config.AddExtensionOption("s3_secret_access_key", "S3 Access Key", LogicalType::VARCHAR);
config.AddExtensionOption("s3_session_token", "S3 Session Token", LogicalType::VARCHAR);
diff --git a/src/include/s3fs.hpp b/src/include/s3fs.hpp
index 525e0dd..a7e933e 100644
--- a/src/include/s3fs.hpp
+++ b/src/include/s3fs.hpp
@@ -231,7 +231,7 @@ public:
return true;
}

- static string GetS3BadRequestError(S3AuthParams &s3_auth_params);
+ static string GetS3BadRequestError(S3AuthParams &s3_auth_params, string correct_region = "");
static string GetS3AuthError(S3AuthParams &s3_auth_params);
static string GetGCSAuthError(S3AuthParams &s3_auth_params);
static HTTPException GetS3Error(S3AuthParams &s3_auth_params, const HTTPResponse &response, const string &url);
diff --git a/src/s3fs.cpp b/src/s3fs.cpp
index cbdecba..72eddc3 100644
index 72eddc3..601ecba 100644
--- a/src/s3fs.cpp
+++ b/src/s3fs.cpp
@@ -872,6 +872,7 @@ void S3FileHandle::Initialize(optional_ptr<FileOpener> opener) {
ErrorData error(ex);
bool refreshed_secret = false;
if (error.Type() == ExceptionType::IO || error.Type() == ExceptionType::HTTP) {
+ // legacy endpoint (no region) returns 400
auto context = opener->TryGetClientContext();
if (context) {
auto transaction = CatalogTransaction::GetSystemCatalogTransaction(*context);
@@ -887,9 +888,13 @@ void S3FileHandle::Initialize(optional_ptr<FileOpener> opener) {
auto &extra_info = error.ExtraInfo();
auto entry = extra_info.find("status_code");
if (entry != extra_info.end()) {
- if (entry->second == "400") {
- // 400: BAD REQUEST
- auto extra_text = S3FileSystem::GetS3BadRequestError(auth_params);
+ if (entry->second == "301" || entry->second == "400") {
+ auto new_region = extra_info.find("header_x-amz-bucket-region");
+ string correct_region = "";
+ if (new_region != extra_info.end()) {
+ correct_region = new_region->second;
+ }
+ auto extra_text = S3FileSystem::GetS3BadRequestError(auth_params, correct_region);
throw Exception(error.Type(), error.RawMessage() + extra_text, extra_info);
@@ -895,7 +895,7 @@ void S3FileHandle::Initialize(optional_ptr<FileOpener> opener) {
correct_region = new_region->second;
}
auto extra_text = S3FileSystem::GetS3BadRequestError(auth_params, correct_region);
- throw Exception(error.Type(), error.RawMessage() + extra_text, extra_info);
+ throw Exception(extra_info, error.Type(), error.RawMessage() + extra_text);
}
if (entry->second == "403") {
@@ -1138,12 +1143,15 @@ bool S3FileSystem::ListFiles(const string &directory, const std::function<void(c
return true;
}

-string S3FileSystem::GetS3BadRequestError(S3AuthParams &s3_auth_params) {
+string S3FileSystem::GetS3BadRequestError(S3AuthParams &s3_auth_params, string correct_region) {
string extra_text = "\n\nBad Request - this can be caused by the S3 region being set incorrectly.";
if (s3_auth_params.region.empty()) {
extra_text += "\n* No region is provided.";
} else {
- extra_text += "\n* Provided region is \"" + s3_auth_params.region + "\"";
+ extra_text += "\n* Provided region is: \"" + s3_auth_params.region + "\"";
+ }
+ if (!correct_region.empty()) {
+ extra_text += "\n* Correct region is: \"" + correct_region + "\"";
// 403: FORBIDDEN
@@ -905,7 +905,7 @@ void S3FileHandle::Initialize(optional_ptr<FileOpener> opener) {
} else {
extra_text = S3FileSystem::GetS3AuthError(auth_params);
}
- throw Exception(error.Type(), error.RawMessage() + extra_text, extra_info);
+ throw Exception(extra_info, error.Type(), error.RawMessage() + extra_text);
}
}
throw;
@@ -941,13 +941,13 @@ bool S3FileSystem::CanHandleFile(const string &fpath) {
void S3FileSystem::RemoveFile(const string &path, optional_ptr<FileOpener> opener) {
auto handle = OpenFile(path, FileFlags::FILE_FLAGS_NULL_IF_NOT_EXISTS, opener);
if (!handle) {
- throw IOException("Could not remove file \"%s\": %s", {{"errno", "404"}}, path, "No such file or directory");
+ throw IOException({{"errno", "404"}}, "Could not remove file \"%s\": %s", path, "No such file or directory");
}

auto &s3fh = handle->Cast<S3FileHandle>();
auto res = DeleteRequest(*handle, s3fh.path, {});
if (res->status != HTTPStatusCode::OK_200 && res->status != HTTPStatusCode::NoContent_204) {
- throw IOException("Could not remove file \"%s\": %s", {{"errno", to_string(static_cast<int>(res->status))}},
+ throw IOException({{"errno", to_string(static_cast<int>(res->status))}}, "Could not remove file \"%s\": %s",
path, res->GetError());
}
return extra_text;
}
diff --git a/test/sql/copy/csv/test_csv_remote.test b/test/sql/copy/csv/test_csv_remote.test
index 4144082..9e51585 100644
--- a/test/sql/copy/csv/test_csv_remote.test
+++ b/test/sql/copy/csv/test_csv_remote.test
@@ -7,15 +7,26 @@ require httpfs
statement ok
PRAGMA enable_verification

+# Test load from url with query string
+query IIIIIIIIIIII
+FROM sniff_csv('https://github.com/duckdb/duckdb/raw/main/data/csv/customer.csv?v=1')
+----
+, " (empty) \n (empty) 0 0 [{'name': column00, 'type': BIGINT}, {'name': column01, 'type': VARCHAR}, {'name': column02, 'type': BIGINT}, {'name': column03, 'type': BIGINT}, {'name': column04, 'type': BIGINT}, {'name': column05, 'type': BIGINT}, {'name': column06, 'type': BIGINT}, {'name': column07, 'type': VARCHAR}, {'name': column08, 'type': VARCHAR}, {'name': column09, 'type': VARCHAR}, {'name': column10, 'type': VARCHAR}, {'name': column11, 'type': BIGINT}, {'name': column12, 'type': BIGINT}, {'name': column13, 'type': BIGINT}, {'name': column14, 'type': VARCHAR}, {'name': column15, 'type': VARCHAR}, {'name': column16, 'type': VARCHAR}, {'name': column17, 'type': BIGINT}] NULL NULL NULL FROM read_csv('https://github.com/duckdb/duckdb/raw/main/data/csv/customer.csv?v=1', auto_detect=false, delim=',', quote='"', escape='', new_line='\n', skip=0, comment='', header=false, columns={'column00': 'BIGINT', 'column01': 'VARCHAR', 'column02': 'BIGINT', 'column03': 'BIGINT', 'column04': 'BIGINT', 'column05': 'BIGINT', 'column06': 'BIGINT', 'column07': 'VARCHAR', 'column08': 'VARCHAR', 'column09': 'VARCHAR', 'column10': 'VARCHAR', 'column11': 'BIGINT', 'column12': 'BIGINT', 'column13': 'BIGINT', 'column14': 'VARCHAR', 'column15': 'VARCHAR', 'column16': 'VARCHAR', 'column17': 'BIGINT'});
+
+
+# This test abuses the LOCAL_EXTENSION_REPO env to make sure tests are only run when running extension tests
+# in duckdb/duckdb. Otherwise you need to pass a data dir when exex
+
+require-env LOCAL_EXTENSION_REPO

# regular csv file
query ITTTIITITTIIII nosort webpagecsv
-SELECT * FROM read_csv_auto('duckdb/data/csv/real/web_page.csv') ORDER BY 1;
+SELECT * FROM read_csv_auto('data/csv/real/web_page.csv') ORDER BY 1;
----

# file with gzip
query IIIIIIIIIIIIIII nosort lineitemcsv
-SELECT * FROM read_csv_auto('duckdb/data/csv/lineitem1k.tbl.gz') ORDER BY ALL;
+SELECT * FROM read_csv_auto('data/csv/lineitem1k.tbl.gz') ORDER BY ALL;
----

query ITTTIITITTIIII nosort webpagecsv
@@ -25,10 +36,3 @@ SELECT * FROM read_csv_auto('https://raw.githubusercontent.com/duckdb/duckdb/mai
query IIIIIIIIIIIIIII nosort lineitemcsv
select * from read_csv_auto('https://raw.githubusercontent.com/duckdb/duckdb/main/data/csv/lineitem1k.tbl.gz') ORDER BY ALL;
----
-
-
-# Test load from url with query string
-query IIIIIIIIIIII
-FROM sniff_csv('https://github.com/duckdb/duckdb/raw/main/data/csv/customer.csv?v=1')
-----
-, " (empty) \n (empty) 0 0 [{'name': column00, 'type': BIGINT}, {'name': column01, 'type': VARCHAR}, {'name': column02, 'type': BIGINT}, {'name': column03, 'type': BIGINT}, {'name': column04, 'type': BIGINT}, {'name': column05, 'type': BIGINT}, {'name': column06, 'type': BIGINT}, {'name': column07, 'type': VARCHAR}, {'name': column08, 'type': VARCHAR}, {'name': column09, 'type': VARCHAR}, {'name': column10, 'type': VARCHAR}, {'name': column11, 'type': BIGINT}, {'name': column12, 'type': BIGINT}, {'name': column13, 'type': BIGINT}, {'name': column14, 'type': VARCHAR}, {'name': column15, 'type': VARCHAR}, {'name': column16, 'type': VARCHAR}, {'name': column17, 'type': BIGINT}] NULL NULL NULL FROM read_csv('https://github.com/duckdb/duckdb/raw/main/data/csv/customer.csv?v=1', auto_detect=false, delim=',', quote='"', escape='', new_line='\n', skip=0, comment='', header=false, columns={'column00': 'BIGINT', 'column01': 'VARCHAR', 'column02': 'BIGINT', 'column03': 'BIGINT', 'column04': 'BIGINT', 'column05': 'BIGINT', 'column06': 'BIGINT', 'column07': 'VARCHAR', 'column08': 'VARCHAR', 'column09': 'VARCHAR', 'column10': 'VARCHAR', 'column11': 'BIGINT', 'column12': 'BIGINT', 'column13': 'BIGINT', 'column14': 'VARCHAR', 'column15': 'VARCHAR', 'column16': 'VARCHAR', 'column17': 'BIGINT'});
diff --git a/test/sql/copy/s3/url_encode.test b/test/sql/copy/s3/url_encode.test
index 66cbd5c..f5a5912 100644
--- a/test/sql/copy/s3/url_encode.test
+++ b/test/sql/copy/s3/url_encode.test
@@ -132,12 +132,20 @@ set s3_endpoint='';
statement error
SELECT * FROM 's3://test-bucket/whatever.parquet';
----
-<REGEX>:.*Unknown error for HTTP HEAD to 'http://test-bucket.s3.eu-west-1.amazonaws.com/whatever.parquet'.*
+<REGEX>:.*HTTP Error: Unable to connect to URL .*http://test-bucket.s3.eu-west-1.amazonaws.com/whatever.parquet.*: 301 .Moved Permanently..*
+.*
+.*Bad Request - this can be caused by the S3 region being set incorrectly.*
+.*Provided region is: .eu-west-1.*
+.*Correct region is: .us-east-1.*

statement error
SELECT * FROM 'r2://test-bucket/whatever.parquet';
----
-<REGEX>:.*Unknown error for HTTP HEAD to 'http://test-bucket.s3.eu-west-1.amazonaws.com/whatever.parquet'.*
+<REGEX>:.*HTTP Error: Unable to connect to URL .*http://test-bucket.s3.eu-west-1.amazonaws.com/whatever.parquet.*: 301 .Moved Permanently..*
+.*
+.*Bad Request - this can be caused by the S3 region being set incorrectly.*
+.*Provided region is: .eu-west-1.*
+.*Correct region is: .us-east-1.*

statement error
SELECT * FROM 'gcs://test-bucket/whatever.parquet';
19 changes: 19 additions & 0 deletions .github/patches/extensions/inet/hugeint_fixes.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
diff --git a/src/inet_functions.cpp b/src/inet_functions.cpp
index da92a4c..afa7446 100644
--- a/src/inet_functions.cpp
+++ b/src/inet_functions.cpp
@@ -185,11 +185,12 @@ static INET_TYPE AddImplementation(INET_TYPE ip, hugeint_t val) {
if (val > 0) {
address_out =
AddOperatorOverflowCheck::Operation<uhugeint_t, uhugeint_t, uhugeint_t>(
- address_in, val);
+ address_in, (uhugeint_t)val);
} else {
+ // TODO: this is off for when val is the minimal uhugeint_t value
address_out =
SubtractOperatorOverflowCheck::Operation<uhugeint_t, uhugeint_t,
- uhugeint_t>(address_in, -val);
+ uhugeint_t>(address_in, (uhugeint_t)(-val));
}

if (addr_type == IPAddressType::IP_ADDRESS_V4 &&
Loading
Loading