v1.17.0

• Improvements
  • functions.when/Column.when and Column.otherwise now accept any literal arguments (e.g. String, int, boolean, or null) in addition to Column instances.
  • Add functions.substring overload with support for start position and length arguments.
  • Add functions.lpad overloads to pad with String, or Array[Byte].
  • Add functions.rpad overloads to pad with String, or Array[Byte].
  • Add DataFrame.sort overload with support for variadic arguments.
  • Add DataFrame.show overloads with parameters to control truncation and number of displayed rows.
• New APIs
  • DataFrame.isEmpty
  • functions.try_to_timestamp
  • functions.try_to_date
  • functions.concat_ws_ignore_nulls
  • functions.array_flatten
  • Row.mkString (with overloads for customizable separators and formatting options)
  • StructType.fieldNames (alias for StructType.names)