diff --git a/proto/substrait/algebra.proto b/proto/substrait/algebra.proto index 300d03f90..e18793b25 100644 --- a/proto/substrait/algebra.proto +++ b/proto/substrait/algebra.proto @@ -319,6 +319,21 @@ message FetchRel { // Recommended type for count is int64. Expression count_expr = 6; } + + // List of fields to sort by to retrieve the number of records specified by `count_mode`. + // At least one field MUST be specified when `with_ties` is true. + // + // Note: the output records are in the order of `sorts` if at least one sort field is specified. + // Otherwise, the input orderedness is preserved. + repeated SortField sorts = 7; + + // Whether to yields 'tie' records of the last record specified by `count_mode`. + // Tie is determined by the `sorts` order (i.e., `sorts` field values are the same if records are 'tie'). + // If this is true, `sorts` MUST specify at least one sort field or else the plan is invalid. + // If this is true and there are more input records specified by the limit `N`, defined by `count_mode`, + // then more than `N` records may be returned if there are ties of the `N`th record. + bool with_ties = 8; + substrait.extensions.AdvancedExtension advanced_extension = 10; } diff --git a/site/docs/relations/logical_relations.md b/site/docs/relations/logical_relations.md index c2317a0ca..7e52d8c6e 100644 --- a/site/docs/relations/logical_relations.md +++ b/site/docs/relations/logical_relations.md @@ -349,16 +349,18 @@ The fetch operation eliminates records outside a desired window. Typically corre | -------------------- | --------------------------------------- | | Inputs | 1 | | Outputs | 1 | -| Property Maintenance | Maintains distribution and orderedness. | +| Property Maintenance | Maintains input distribution. Orderedness is `Sort Fields` if specified. Otherwise, input orderedness. | | Direct Output Order | Unchanged from input. | ### Fetch Properties | Property | Description | Required | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------- | -| Input | A relational input, typically with a desired orderedness property. | Required | +| Input | A relational input. | Required | | Offset Expression | An expression which evaluates to a non-negative integer or null (recommended type is `i64`). Declares the offset for retrieval of records. An expression evaluating to null is treated as 0. | Optional, defaults to a 0 literal. | | Count Expression | An expression which evaluates to a non-negative integer or null (recommended type is `i64`). Declares the number of records that should be returned. An expression evaluating to null indicates that all records should be returned. | Optional, defaults to a null literal. | +| Sort Fields | List of one or more sort fields to define a desired orderedness. If 'With Ties' is true then there must be at least one sort field or else the plan is invalid. | Optional, the default will preserve the input orderedness | +| With Ties | Whether to return "tied rows" which are rows equal to last row that would be returned by 'Count Expression'. If false, at most 'Count Expression' records are returned. If true, it may yield more records than 'Count Expression'. | Optional, defaults to false. | === "FetchRel Message"