FR: Modular message footer API

[dlqqq]

Problem

Upstream issue in Jupyter AI: jupyterlab/jupyter-ai#1297

#198 adds a new feature that allows Jupyter Chat to replicate the "Stop streaming" capability in Jupyter AI v2. Thank you all for working on this! 🤗

However, I think that we should also explore other ways of allowing users to stop streaming messages. Users should be able to send messages regardless of whether an AI is streaming a reply to this user. The UI in Jupyter AI v2 made sense because it was just one user w/ one AI back then, but it doesn't scale well to many users w/ many AIs.

Here, I propose building a modular message footer API to:

1. Allow Jupyter AI to provide a "stop streaming" button at the bottom of each chat message in the center, and
2. Allow other labextensions to add something to the bottom of each chat message, without clobbering Jupyter AI's "Stop streaming" button.

The reason I seek 2) is because other labextensions may want to add message footers as well. This feature was added in Jupyter AI 2.x for other applications to provide custom UI elements to each message. For example, some developers have used this API to render thumbs up / thumbs down buttons to allow users to "rate" an AI response.

Proposed design

1. Jupyter Chat optionally requests the ChatMessageFooter token. When other plugins provide this, they have to provide a React component that renders their footer, and pick a location to render. The React component accepts the current message & position as props. The location can be "left", "right", or "center".

export const ChatMessageFooter = new Token<IChatFactory>(
  'jupyterlab-chat:ChatMessageFooter'
);

export interface ChatMessageFooterProps {
  message: Message
  position: 'left' | 'center' | 'right'
}

export interface ChatMesageFooter {
  component: React.FC<ChatMessageFooterProps>
  position: 'left' | 'center' | 'right'
}

2. Jupyter Chat accepts all message footers and builds an object like this:

type Footers = {
  Left?: ChatMessageFooter
  Center?: ChatMessageFooter
  Right?: ChatMessageFooter
}

If 2 plugins provide the same footer at the same location, we can just pick 1 arbitrarily. The idea is that if one labextension is occupying the 'center' position, another labextension can render their footer on the 'left' or 'right' position. Therefore these conflicts should not happen to begin with.

3. Jupyter Chat then renders the footers object above in a flexbox, like this:

<Box sx={{ display: 'flex', justifyContent: 'space-between'}}>
  {footers.Left ? <footers.Left message={message} position="left" /> : <div />
  // ... similarly for Center and Right
</Box>

This design thus allows up to 3 extensions adding 3 different footers, and makes the footer component fully customizable. Jupyter Chat will only control what props are passed to each message footer component.

Additional context

@brichet Jupyter AI also needs some way of setting metadata on each message to indicate whether it is complete or still streaming. We will need this data to know when to show/hide the "Stop streaming" button. Do you know if this is possible in Jupyter Chat?

[brichet]

Thanks @dlqqq for the detailed issue.

Jupyter AI also needs some way of setting metadata on each message to indicate whether it is complete or still streaming. We will need this data to know when to show/hide the "Stop streaming" button. Do you know if this is possible in Jupyter Chat?

Currently the chat is aware of people writing using the awareness object of the YDocument

jupyter-chat/packages/jupyterlab-chat/src/model.ts

Line 246 in 85443f7

awareness.setLocalStateField('isWriting', true);

The same awareness is used in Jupyter AI when streaming a response: https://github.com/jupyterlab/jupyter-ai/blob/304d9f8611b6b7dea25eeb0a45ab94787ea48eef/packages/jupyter-ai/jupyter_ai/chat_handlers/utils/streaming.py#L47.

The type of isWriting property could be a boolean | string, with the string being a message ID when this is an update on a message.

[brichet]

I opened #208 to retrieve the message ID that the writer is editing (between other features).

[brichet]

Also opened #210 to add the message footers.
It is quite similar to the proposed design, except that the token provides a registry, to allow third party extension to add footers.

When other plugins provide this, they have to provide a React component that renders their footer

IIUC, I don't think that this is possible, a token can be provided by only one extension. This is why I choose to provide a registry instead.