Open
Description
The command line behavior is fine, but when using DataFrameConverter
as a library in python code, setting output_columns
effectively does nothing,
https://github.com/DocNow/twarc-csv/blob/main/dataframe_converter.py#L530
because that gets applied in
https://github.com/DocNow/twarc-csv/blob/main/csv_writer.py#L70
instead. Internally, DataFrameConverter
always works with the full set of columns. Not sure if this is desirable or counter intuitive enough to change, it probably should be outputting the restricted set to be less awkward.