allow `to_numpy` conversion on typed output?

[feefladder]

working on #103, solving #87, there were some ffi headaches which made me consider numpy. That's overly complex for the internal Rust part, but it occured to me that converting to numpy on the output would be rather ergonomic and best done in Rust (so in this crate). Currently what's in #103 (will be split in two, this issue is about a third PR, but influences those)

// src/raster_data.rs
pub enum RasterData {
  U8(Vec<u8>),
  U16(Vec<u16>),
  // ..etc
}

impl RasterData {
  fn from_predictor_info(info: PredictorInfo, ..) -> Self
}
// python/src/raster_data.rs
#[pyclass]
struct PyRasterData {
  inner: RasterData
}

#[pymethods]
impl PyRasterData {
  /// Convert this to a numpy array
  /// shape is: (didn't test yet)
  /// [`PlanarConfiguration::Chunky`]: (height, width, channels)
  /// [`PlanarConfiguration::Planar`]: (channels, height, width)
  fn to_numpy(width: u32/usize, height: u32/usize, planar_configuration: PlanarConfiguration) -> NumpyArray {
    // single match statement with trivial conversions to 1-d arrays

    // match on planarconfig and reshape the arrays
  }
}

What's slightly ugly in this api is that those width, height, planar_config are in Tile, but then lost at RasterData, requiring the user to keep those around. So the question is if we want a wrapping DecodedTile struct (maybe only in Python) like:

struct PyDecodedTile {
  data: RasterData,
  width: u32/usize, // whatever it is in Tile or numpy wants, probably usize
  height: u32/usize,
  planar_configuration: PlanarConfiguration
}

or, alternatively, a decode_tile_to_numpy python convenience function that has access to those values anyways - I think that's neat, is the extra dependency and maintenance burden worth it?

[kylebarron]

I think we'd rather prefer a buffer protocol or DLPack interface that doesn't copy data between Rust and Numpy

[feefladder]

this issue

I'm all in on not copying data, hencewhy I closed #103, because it needed more thought. I think we can be zero-copy python <-> rust purely with ndarray and numpy (keeping non-optional dependencies low). Once that memory is managed by Python, it is also reference-counted. Strictly this issue (Rust enum{Vec<T>} -> python`) would be a:

fn to_numpy(&self) -> PyAny {
  match self {
    U8(v) => v.into_pyarray(), // PyArray<u8>
    U16(v) => v.into_pyarray(), // PyArray<u16>
    _ => todo!(),
  }
}

This is zero-copy, the only caveat is that it cannot be resized python-side afterwards.

going the other way

If we'd also want to accept this, I think we'd only ever work on byte data (e.g. the unpredicting):

fn to_u8_mut_slice(array: PyAny) -> AsyncTiffResult(&mut[u8]) {
  match array.getattr("dtype)? {
    "float64" => Ok(bytemuck::cast_slice_mut::<u8>(array.downcast::<PyArrayDyn<f64>>().unwrap().as_slice_mut()?))
  }
}

would work. Then this could be passed to predictors. Or even some enum{&mut[T]}.

The problem of #103 is (still) making the signature of the Decoder trait python-friendly, while also not requiring any Python types/dependencies for Rust users.... Indeed maybe DLPack could work, but I think it could be made to work nicely in this direction as well... So, what we'd need is a Python-managed, dynamic vector data type that is also light-weight and rust-friendly... (or accept the to-typed copy in case of non-float predictors, keeping the Decoder signature a -> Bytes)

saving more copies

The complexity in #103 was when I was trying to save a copy by moving this step earlier into the decoding process.

On copies:
There is a copy needed to go from an unaligned Bytes::bytes (which is like an Arc<Vec<u8>>) to something typed (like an Arc<Rasterdata> which is an enum over Arc<Vec<f64>>) - unless we'd do unsafe pointer checks (no-go imo, even bytemuck doesn't Vec<u8>->Vec<f64>). That is purely a Rust-side problem.

Then in #103, I thought to put the typing copy at the decompression step, since that is rarely zero-copy (except for uncompressed data). Then another copy is needed in the case of floating-point predictor (in the case of padding, the output array is shorter. Also the algorithm needs a copy).

Since -in #103- the internal, rust-side decode function from the registry works on a &mut[u8] result_buffer, it's rather agnostic[^1] to however that memory is managed (Bytes, ndarray, etc).

I think we can be single-copy[^2] for the entire decoding step with a python decompressor if we:

1. keep the Rust-side decode signature like fn decode(in_buf: Bytes, out_buf: &mut[u8], predictor_info: PredictorInfo)
2. take a numpy array and bytemuck a slice to this &mut[u8]

[^1]: If working with numpy, which is ndarray, there are strides and the possiibility of non-contiguous data. I think we can safely error at that point. [^2]: except for float predictor

[feefladder]

Coming back to this, I think we can:

• expose some function to Python that allows creating a properly sized enum{Vec<T>}
• make the Decoder::decode(in: Bytes) -> enum{Vec<T>}

so a user cannot convert this from a PyObject... (we cannot move it to owned)

Also a Decoder::decode(in: Bytes) -> enum{&mut[T]} doesn't work because Rust can't create a reference that outlives a function (Python can)
And Decocer::decode(in: Bytes, out: &mut[u8]) doesn't work, because &mut[u8] can't be passed to a Python function.

So one sad compromise is:

fn decode_tile(&self, registry) -> enum{Vec<T>} { // where enum{Vec<T>} does zero-copy -> numpy array
  let uncompressed: Bytes = registry[self.compression].decompress();
  match self.predictor {
    None => {
      let res: enum{Vec<T>} = uncompressed.clone(); // explicit copy
      fix_endianness(res, self.endianness)
      res
    },
    Horizontal => {
      // also "needless" copy
    },
    Float => {
      let res: enum{Vec<T>} = RasterData::Calc(PredictorInfo);
      // do floating point prediction into res, which now doesn't copy
    }
  }
}

// python/src/tile
fn decode_tile_to_numpy(&self, info: PredictorInfo) -> PyAny {
  let res = self.decode_tile(..);
  // convert Vec to PyArray
  let arr = res.to_numpy();
  arr.reshape(/*info based on PlanarConfig*/)
}