Managing sessions across multiple SSE MCP servers

[dronnix]

Hi, MCP developers!

How do you manage client sessions across multiple MCP server instances? We need to be sure that the events and messages connections will be established with the same instance. Do you use sticky sessions on the infrastructure level, or are other solutions possible?

[rwjblue-glean]

At the moment, we are using session affinity (aka sticky sessions) for this -- which seems to be working. I did take an initial stab at making an abstraction for better horizontal scaling wihtout session affinity in #186 but I haven't had a chance to circle back to that PR in a little bit 😩 (and not sure when I will -- given that session affinity is working for us atm).

[dronnix]

@robert-jackson-glean Thanks! Is it affinity by IP, cookies, or something else?

[rwjblue-glean]

@dronnix - Ya, you definitely can not use cookie based affinity (the cookies aren't shared across requests from the various clients). Using IP based works well for the most part (it does still have issues when many clients are behind a single NAT or when NAT'ed behind multiple public IPs), but I do think I'll need to actually update and finish off #186 to make it the most stable.

[SamMorrowDrums]

Hi thanks for the info @robert-jackson-glean, we are looking into building a remote server at GitHub. For full resilience across multiple datacenters, we are having to look at using:

• external session store
• being able to handle request for a single session across multiple servers (and likely we will have to wire them up via a queue to make the server sent events work).

We are expecting to have to fork mcp-go to enable experimenting with that, and I'm curious if it will be something you want to merge upstream or not.

One thing that I have been mulling over, is that the session management and session tools probably took the wrong track and (as is proposed in Google's Go SDK, I am beginning to feel that composition is a nicer approach.

That would mean for stateless use-cases people can use the same server instance, but where servers present different tools to different users, it would mean creating a new instance per request. Sessions would also need to be managed by the implementer, not mcp-go, so that it's possible to do things like have a user session split between two servers one with the long-running request, and another receiving the tool calls etc. (as is the case with streamable http if I understand correctly).

The quote from the Go SDK:

Importantly, since they serve many connections, the HTTP handlers must accept a callback to get an MCP server for each new session. As described below, MCP servers can optionally connect to multiple clients. This allows customization of per-session servers: if the MCP server is stateless, the user can return the same MCP server for each connection. On the other hand, if any per-session customization is required, it is possible by returning a different Server instance for each connection.

I hope eventually running off a single connection without session affinity (via websockets or something) can replace this, but in the short term we have very limited options given global scale 😅

Do you have any thoughts on this, especially how we could best achieve this with mcp-go?

[SamMorrowDrums]

I mean other than #186 which I guess is exactly where you are going with this.