Zero-copy bindings?

[edsko]

The marshaling code generated by hs-bindgen is based around Storable: we do a full copy of the data on entry to and on exit from C. One could imagine generating different kind of code, where we deal with mutable types instead, still providing ways to access parts of a larger structure (fields of a struct, say), but without the copying.

This is just a placeholder issue at the moment, marked as low-priority; we're not currently actively planning to do this. But it's perhaps worth thinking about.

[edsko]

Possibly relevant/useful: https://hackage.haskell.org/package/mutable

[edsko]

The more I think about this, the more I think that this would both not be terribly difficult and rather useful.

For full demo of the code below, see #816 .

Motivation

Consider the following C declarations:

struct point {
    int x;
    int y;
};

The code generated by hs-bindgen (cleaned up a bit) looks like this:

data Point = Point {
      x :: CInt
    , y :: CInt
    }
  deriving stock (Show, Eq)

instance Storable Point where
  sizeOf    _ = 8
  alignment _ = 4

  peek ptr =
          pure Point
      <*> peekByteOff ptr 0
      <*> peekByteOff ptr 4

  poke ptr Point{x, y} = do
      pokeByteOff ptr 0 x
      pokeByteOff ptr 4 y

The problem is that this Storable instance knows that the offset of field y is 4, but this knowledge is not available to users of hs-bindgen.

Proposal

We could (should?) instead do the following. First, define a class that abstracts over fields of C structs:

class Storable (FieldType a field) => HasCField (a :: Type) (field :: Symbol) where
  type FieldType a field :: Type
  ptrToField :: Proxy field -> Ptr a -> Ptr (FieldType a field)

The instances for Point would be

instance HasCField Point "x" where
  type FieldType Point "x" = CInt
  ptrToField _ ptr = ptr `plusPtr` 0

instance HasCField Point "y" where
  type FieldType Point "y" = CInt
  ptrToField _ ptr = ptr `plusPtr` 4

We can define two helper functions for peeking and poking specific fields:

pokeField :: HasCField a field => Proxy field -> Ptr a -> FieldType a field -> IO ()
pokeField field ptr val = poke (ptrToField field ptr) val

peekField :: HasCField a field => Proxy field -> Ptr a -> IO (FieldType a field)
peekField field ptr = peek (ptrToField field ptr)

This would then allow us to rewrite the Storable instance like this:

instance Storable Point where
  sizeOf    _ = 8
  alignment _ = 4

  peek ptr =
          pure Point
      <*> peekField (Proxy @"x") ptr
      <*> peekField (Proxy @"y") ptr

  poke ptr Point{x, y} = do
      pokeField (Proxy @"x") ptr x
      pokeField (Proxy @"y") ptr y

One minor benefit is that this definition of peek actually refers to specific fields, rather than (just) depending on positional fields; slightly more readable. But the more important benefit is that users now have access to these offsets, making it possible to modify parts of structs, even potentially nested. For example, if we also have

struct rect {
    struct point topleft;
    struct point bottomright;
};

void show_rect(struct rect* r);

we'd end up with HasCField instances such as this:

instance HasCField Rect "topleft" where
  type FieldType Rect "topleft" = Point
  ptrToField _ ptr = ptr `plusPtr` 0

OverloadedRecordDot

We can even integrate this with HasField:

instance (HasCField Point field, ty ~ FieldType Point field)
      => GHC.HasField field (Ptr Point) (Ptr ty) where
  getField = ptrToField (Proxy @field)

instance (HasCField Rect field, ty ~ FieldType Rect field)
      => GHC.HasField field (Ptr Rect) (Ptr ty) where
  getField = ptrToField (Proxy @field)

This then allows for very succinct (and idiomatic) updates of fields of records:

with (Rect (Point 1 2) (Point 3 4)) $ \ptr -> do
  poke ptr.bottomright.x 99
  show_rect ptr

Which I think is very nice, and could be rather useful indeed. So I think we should implement this; it's not a huge change to what we already do, and would give us a more expressive API.

[edsko]

Credit where credit is due, bindings-DSL provides these offsets as well, albeit as simple functions. See discussion of p'regex_t're_nsub in the bindings-DSL tutorial.

[edsko]

We've decided to go ahead and do this.

[edsko]

Unions

This scales nicely to unions as well. Given

union pointVsArray {
    struct point asPoint;
    int asArray[2];
};

This results in the usual bindings:

newtype PointVsArray -- opaque

get_pointVsArray_asArray :: PointVsArray -> ConstantArray 2 CInt
get_pointVsArray_asPoint :: PointVsArray -> Point
set_pointVsArray_asArray :: ConstantArray 2 CInt -> PointVsArray
set_pointVsArray_asPoint :: Point                -> PointVsArray

we can define

instance HasCField PointVsArray "asPoint" where
  type FieldType PointVsArray "asPoint" = Point
  ptrToField _ = castPtr

instance HasCField PointVsArray "asArray" where
  type FieldType PointVsArray "asArray" = ConstantArray 2 CInt
  ptrToField _ = castPtr

instance (HasCField PointVsArray field, ty ~ FieldType PointVsArray field)
      => GHC.HasField field (Ptr PointVsArray) (Ptr ty) where
  getField = ptrToField (Proxy @field)

(Just like for structs, these can probably also be used in the definition of the getters and setters, though I haven't fully worked that out yet.) This is especially useful for unions like this which provide an alternative view on the same data; in such cases we probably don't want to copy, and now we don't have to:

with (Z.set_pointVsArray_asArray $ CA.fromList [55, 66]) $ \ptr -> do
  y <- peek ptr.asPoint.y
  print y