Skip to content

v1.17.0

Choose a tag to compare

@sfc-gh-heshah sfc-gh-heshah released this 16 Sep 21:54
· 16 commits to main since this release
  • Improvements
    • functions.when/Column.when and Column.otherwise now accept any literal arguments (e.g. String, int, boolean, or null) in addition to Column instances.
    • Add functions.substring overload with support for start position and length arguments.
    • Add functions.lpad overloads to pad with String, or Array[Byte].
    • Add functions.rpad overloads to pad with String, or Array[Byte].
    • Add DataFrame.sort overload with support for variadic arguments.
    • Add DataFrame.show overloads with parameters to control truncation and number of displayed rows.
  • New APIs
    • DataFrame.isEmpty
    • functions.try_to_timestamp
    • functions.try_to_date
    • functions.concat_ws_ignore_nulls
    • functions.array_flatten
    • Row.mkString (with overloads for customizable separators and formatting options)
    • StructType.fieldNames (alias for StructType.names)