Add support for DuckDB MAP functions #902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

shivampr wants to merge 3 commits into duckdb:main from shivampr:map-functions-implementations

+707 −1

shivampr commented Aug 27, 2025 •

edited

Loading

Changes

Implement map_extract, map_keys, map_values, cardinality, element_at, map_concat, map_contains, map_contains_entry, map_contains_value, map_entries, map_extract_value, and map_from_entries functions

Testing

All 61 regression tests pass (make installcheck)
Manually verified all functions in local PostgreSQL instance

Closes #774

JelteF requested changes

View reviewed changes

Collaborator

JelteF left a comment

Thank you for your work on this! The functions you defined had a bunch of wrong types. And the docs are clearly fully AI generated, with no manual sanity check, because the syntax doesn't work.

sql/pg_duckdb--1.0.0--1.1.0.sql

Comment on lines +3 to +9

+              CREATE FUNCTION @[email protected]_extract(map_col duckdb.map, key text)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_extract(map_col duckdb.unresolved_type, key text)

Collaborator

JelteF Sep 8, 2025

Suggested change

      
            CREATE FUNCTION @[email protected]_extract(map_col duckdb.map, key text)
          
            RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
          
            CREATE FUNCTION @[email protected]_extract(map_col duckdb.unresolved_type, key text)
          
            CREATE FUNCTION @[email protected]_extract(map_col duckdb.map, key any)
          
            RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
          
            CREATE FUNCTION @[email protected]_extract(map_col duckdb.unresolved_type, key any)

Author

shivampr Oct 9, 2025 •

edited

Loading

PostgreSQL threw ERROR: syntax error at or near "any" when i actually used this.Thus used duckdb.unresolved_type + explicit type overloads instead.

Please let me know if this makes sense.

sql/pg_duckdb--1.0.0--1.1.0.sql Outdated

Comment on lines 21 to 24

+              CREATE FUNCTION @[email protected](map_col duckdb.map)
+              RETURNS integer AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected](map_col duckdb.unresolved_type)
+              RETURNS integer AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

Collaborator

JelteF Sep 8, 2025

Suggested change

      
            CREATE FUNCTION @[email protected](map_col duckdb.map)
          
            RETURNS integer AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
          
            CREATE FUNCTION @[email protected](map_col duckdb.unresolved_type)
          
            RETURNS integer AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
          
            CREATE FUNCTION @[email protected](map_col duckdb.map)
          
            RETURNS numeric AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
          
            CREATE FUNCTION @[email protected](map_col duckdb.unresolved_type)
          
            RETURNS numeric AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

sql/pg_duckdb--1.0.0--1.1.0.sql

Comment on lines +27 to +50

+              CREATE FUNCTION @[email protected]_at(map_col duckdb.map, key text)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_at(map_col duckdb.unresolved_type, key text)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_at(map_col duckdb.map, key integer)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_at(map_col duckdb.unresolved_type, key integer)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

Collaborator

JelteF Sep 8, 2025

Suggested change

-CREATE FUNCTION @[email protected]_at(map_col duckdb.map, key text)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_at(map_col duckdb.unresolved_type, key text)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_at(map_col duckdb.map, key integer)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_at(map_col duckdb.unresolved_type, key integer)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+CREATE FUNCTION @[email protected]_at(map_col duckdb.map, key any)
+RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+CREATE FUNCTION @[email protected]_at(map_col duckdb.unresolved_type, key any)
+RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

sql/pg_duckdb--1.0.0--1.1.0.sql Outdated Show resolved Hide resolved

sql/pg_duckdb--1.0.0--1.1.0.sql

Comment on lines +43 to +78

+              CREATE FUNCTION @[email protected]_contains(map_col duckdb.map, key text)
+              RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_contains(map_col duckdb.unresolved_type, key text)
+              RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_contains(map_col duckdb.map, key integer)
+              RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_contains(map_col duckdb.unresolved_type, key integer)
+              RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

Collaborator

JelteF Sep 8, 2025

Suggested change

-CREATE FUNCTION @[email protected]_contains(map_col duckdb.map, key text)
-RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_contains(map_col duckdb.unresolved_type, key text)
-RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_contains(map_col duckdb.map, key integer)
-RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_contains(map_col duckdb.unresolved_type, key integer)
-RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+CREATE FUNCTION @[email protected]_contains(map_col duckdb.map, key any)
+RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+CREATE FUNCTION @[email protected]_contains(map_col duckdb.unresolved_type, key any)
+RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

sql/pg_duckdb--1.0.0--1.1.0.sql

Comment on lines +59 to +102

+              CREATE FUNCTION @[email protected]_contains_value(map_col duckdb.map, value text)
+              RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_contains_value(map_col duckdb.unresolved_type, value text)
+              RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

Collaborator

JelteF Sep 8, 2025

Suggested change

      
            CREATE FUNCTION @[email protected]_contains_value(map_col duckdb.map, value text)
          
            RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
          
            CREATE FUNCTION @[email protected]_contains_value(map_col duckdb.unresolved_type, value text)
          
            RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
          
            CREATE FUNCTION @[email protected]_contains_value(map_col duckdb.map, value any)
          
            RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
          
            CREATE FUNCTION @[email protected]_contains_value(map_col duckdb.unresolved_type, value any)
          
            RETURNS boolean AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

sql/pg_duckdb--1.0.0--1.1.0.sql

Comment on lines +71 to +130

+              CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.map, key text)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.unresolved_type, key text)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.map, key integer)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+              CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.unresolved_type, key integer)
+              RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

Collaborator

JelteF Sep 8, 2025

Suggested change

-CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.map, key text)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.unresolved_type, key text)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.map, key integer)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
-CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.unresolved_type, key integer)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.map, key any)
+RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+CREATE FUNCTION @[email protected]_extract_value(map_col duckdb.unresolved_type, key any)
+RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

sql/pg_duckdb--1.0.0--1.1.0.sql Outdated Show resolved Hide resolved

sql/pg_duckdb--1.0.0--1.1.0.sql Outdated

Comment on lines 81 to 82

		CREATE FUNCTION @[email protected]_from_entries(entries duckdb.unresolved_type)
		RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

Collaborator

JelteF Sep 8, 2025

Suggested change

-CREATE FUNCTION @[email protected]_from_entries(entries duckdb.unresolved_type)
-RETURNS duckdb.unresolved_type AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+CREATE FUNCTION @[email protected]_from_entries(entries duckdb.struct[])
+RETURNS duckdb.map AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;
+CREATE FUNCTION @[email protected]_from_entries(entries duckdb.unresolved_type)
+RETURNS duckdb.map AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

docs/functions.md Outdated Show resolved Hide resolved

Collaborator

JelteF commented Oct 8, 2025

@shivampr Are you intending to address the comments?

shivampr added 2 commits

October 8, 2025 18:16


          Add support for DuckDB MAP functions

ae07316

- Add map_extract, map_keys, map_values, cardinality, element_at, map_concat,
  map_contains, map_contains_entry, map_contains_value, map_entries,
  map_extract_value, and map_from_entries functions
- Create migration file pg_duckdb--1.0.0--1.1.0.sql with function definitions
- Update metadata cache to recognize all MAP functions as DuckDB-only
- Add comprehensive tests covering all functions with edge cases
- Update documentation with detailed usage examples and function descriptions
- Bump version to 1.1.0 in pg_duckdb.control
- Add regression tests to schedule

Closes duckdb#774


          Fix MAP functions per review feedback

e3f6a7b

Use duckdb.unresolved_type with explicit overloads for type flexibility,
fix return types, update docs to use duckdb.query() syntax.

Closes duckdb#774

shivampr force-pushed the map-functions-implementations branch from adeee94 to e3f6a7b Compare

October 9, 2025 02:35


          Resolved merge conflicts

281ff44

Closes duckdb#774

shivampr requested a review from JelteF

October 9, 2025 06:05

Author

shivampr commented Oct 9, 2025

@shivampr Are you intending to address the comments?

Yes, apologies for the delay! I've tried to address all your comments.
Thanks for the detailed feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet