Server-side shared query cache

[PepijnSenders]

Hoping that this hasn't been discussed yet, but I've run into a design problem with the new hydration API.

The issue

Say we have a node.js server with a shared singleton queryCache object like:

import express from "express";
import { makeQueryCache, QueryCache } from "react-query";
import { dehydrate } from "react-query/hydration";

const queryCache: QueryCache = makeQueryCache();

const server = express();

server.get("/a", async (req, res) => {
  await queryCache.prefetchQuery("a", () => Promise.resolve("A"));

  res.send(dehydrate(queryCache));
});

server.get("/b", async (req, res) => {
  await queryCache.prefetchQuery("b", () => Promise.resolve("B"));

  res.send(dehydrate(queryCache));
});

server.listen(3000);

As you can probably expect either endpoint will return their cached query, but if you go to the other page it will also start responding with the other query.

After requesting both endpoints, both endpoints will now return this JSON:

{
	"queries": [{
		"config": {},
		"queryKey": ["a"],
		"updatedAt": 1599832897654,
		"data": "A"
	}, {
		"config": {},
		"queryKey": ["b"],
		"updatedAt": 1599832895728,
		"data": "B"
	}]
}

This doesn't matter if your response is just B, but if you have a really big app, like we do, which also runs for different countries and markets, we need to filter this in some way. At the moment we have something like this to filter it per country and locale:

import express from "express";
import { makeQueryCache, QueryCache, QueryKey } from "react-query";
import { dehydrate } from "react-query/hydration";

const queryCache: QueryCache = makeQueryCache();

const server = express();

type LocalizedParams = QueryKey & {
  country: string;
};

const dehydrateCache = (country: string) => {
  return dehydrate(queryCache, {
    shouldDehydrate: (query) => {
      const [, params] = query.queryKey;

      if (params && (params as LocalizedParams).country === country) {
        return true;
      }

      return false;
    },
  });
};

server.get("/a", async (req, res) => {
  await queryCache.prefetchQuery(["a", { country: "US" }], () =>
    Promise.resolve("A")
  );

  res.send(dehydrateCache("US"));
});

server.get("/b", async (req, res) => {
  await queryCache.prefetchQuery(["b", { country: "DE" }], () =>
    Promise.resolve("B")
  );

  res.send(dehydrateCache("DE"));
});

server.listen(3000);

As you can see this already becomes a bit more complex, we need to use the shouldDehydrate function (which is a great API in itself by the way) to filter the queries. All the queries now need to have the same signature and it would break if someone uses a query that is just a string, or maybe is the same for all countries.

Possible solution

I hope someone else thought of something more elegant for this, but my idea would be to do something like this:

import express from "express";
import { makeQueryCache, QueryCache, QueryKey } from "react-query";
import { dehydrate } from "react-query/hydration";
import deepEqual from "fast-deep-equal";

const queryCache: QueryCache = makeQueryCache();

const server = express();

let requestLog: QueryKey[] = [];

const dehydrateCache = () => {
  return dehydrate(queryCache, {
    shouldDehydrate: (query) => {
      const hasBeenRequested = requestLog.find((params) => {
        return deepEqual(params, query.queryKey);
      });

      return !!hasBeenRequested;
    },
  });
};

server.use((req, res, next) => {
  requestLog = [];

  next();
});

server.get("/a", async (req, res) => {
  const params = ["a", { country: "US" }];
  requestLog.push(params);
  await queryCache.prefetchQuery(params, () => Promise.resolve("A"));

  res.send(dehydrateCache());
});

server.get("/b", async (req, res) => {
  const params = ["b", { country: "US" }];
  requestLog.push(params);
  await queryCache.prefetchQuery(params, () => Promise.resolve("B"));

  res.send(dehydrateCache());
});

server.listen(3000);

In this example I need to keep a log of all the requests per request handler. Then when I dehydrate I check which requests were done and I reset the list of requests every time a request happens.

@tannerlinsley any ideas, maybe this could be something that's integrated inside react-query?

[Ephem]

Thanks for the feedback, this is an interesting use case! So what you need is a way to keep track of which queries were fetched in this particular request so that you can dehydrate only those? Sounds like a very classic SSR problem. 😁

I think the solution you came up with is a good one. 😄 I guess what you could do to simplify it a bit is to wrap prefetchQuery with your own function that takes care of the requestLog-stuff behind the scenes.

A slightly different/expanded approach that could be generalized into a third party library would be to create a kind of full proxy for that shared cache, and in that proxy keep track of which queries has been accessed. Something like const proxyCache = createProxyCache(queryCache) and expose a proxyCache.usedQueryKeys from that proxy. All methods on that proxy would work as normal and towards the original cache, with the addition that they keep track of the query keys used. The only thing I think we could do differently than this in React Query directly is to make dehydrate automatically only dehydrate the queries that where used by the proxy cache instead of exposing a proxyCache.usedQueryKeys.

While definitely a great case, my initial feeling is that since the hydration APIs are new, we want to keep the API surface as small as possible to leave ourselves room to figure out the best patterns to support going forward. I think there are probably a bunch of cool things we could do to better support shared caches in the future. 😄

An example idea for the future. Maybe you could take the concept above and pair it with some way of marking queries "shared"? Any queries not explicitly marked "shared" would only be cached in the "proxy" (which now is something other than a proxy) and thus exist for that request only, while any queries marked "shared" would be cached in the original shared cache. It's because of ideas like this I'm hesitant implementing stuff too early before we've figured out the bigger picture, I especially think there is a lot of valuable feedback and experimentation people like you and others in the ecosystem can help provide to form that!

[PepijnSenders]

Thanks for the answer @Ephem! Was basically also asking because I didn't want to re-implement something that has already been done. I also really like the proxy idea, that might be a bit more clean than having to manage a completely different entity (the requestQueue).

[MellKam]

Facing the same problem five years later. Is there a better solution available now?

[escaton]

I tried the combination of fresh queryClient on each request, passing a shared instance of experimental_createQueryPersister in defaultOptions for all queries, and then dehydrating that queryClient. Seems working: the cache is shared between requests, and the dehydrated queries are only ones accessed in the lifetime of the queryClient.