[v3.0.0ax] Restore the "stop streaming" feature

[brichet]

Problem

The "stop streaming" feature (issue and PR) has not been ported yet in V3.

I haven't found any issue about that, so I wonder if we want it back in V3 with the same design/behavior as previously (in V2).

Proposed Solution

Restore the stop streaming feature in jupyter-ai V3.

Additional context

It should now be possible to restore it using jupyterlab-chat 0.9.0, which includes a way to customize the input toolbar: jupyterlab/jupyter-chat#198.

[dlqqq]

@brichet Thanks for opening this issue & for making the upstream changes to Jupyter Chat!

@jtpio had called out that the current release plan lacks milestones & clear guidance for people to contribute. I've put together a v3.0.0ax milestone. This will track all regressions relative to v2 that we should address by the last alpha release, like this issue here. I've added this issue to the v3.0.0ax milestone for tracking.

[dlqqq]

@brichet Thank you for working on this! Summarizing some of the discussion we had this morning here:

We should explore putting the "stop streaming" button at the bottom of the AI message itself instead of replacing the Send button. Jupyter AI should work just like any other chat application, so whether the AI is still writing a response shouldn't prevent a user from sending another message.

Also, we should reach alignment on the API that Jupyter Chat will provide for this, since "Stop streaming" won't be applicable to users who aren't using Jupyter AI. That means that we'll probably have to build this in a modular fashion, i.e. Jupyter AI has to provide a plugin that somehow defines the frontend as well.

[dlqqq]

@brichet I've opened a new issue to discuss this in Jupyter Chat: jupyterlab/jupyter-chat#202

[dlqqq]

@3coins I think this would be a good issue to look into while I am away. Here's a high-level summary:

• Nicolas has already implemented the required changes in Jupyter Chat: Add a message footer registry, to allow extensions to provides their footers jupyter-chat#210

• A PR for this should be done on top of the branch in Introduce AI persona framework #1324. We currently plan to keep Introduce AI persona framework #1324 open for another week to let others give feedback. I'll merge it after I'm back.

• The PR should add a "Stop streaming" button as a message footer. The button should only appear under messages from AI personas that are still streaming.

• On click, it should call some new API endpoint (e.g. POST /api/ai/stream/{stream_id}). That endpoint should stop an AI stream & suffix the message with a tombstone like "(AI response halted by user)."

[andrii-i]

jupyterlab/jupyter-chat#210 is now merged and published as Jupyter Chat v0.11.0 https://github.com/jupyterlab/jupyter-chat/releases/tag/v0.11.0.