Skip to content

Conversation

LukeButters
Copy link
Contributor

@LukeButters LukeButters commented Oct 7, 2025

Background

Most Halibut tests have attributed like: LatestClientAndLatestServiceTestCases which result in that test running under a range of combinations e.g. listening, polling, polling web sockets, good networks, networks with latency etc.

With chance we add another combination which is to test polling with the Redis Pending Request Queue.

Doing so means we get to exercise the queue under a range of different conditions doing so has revealed two flaws in the queue:

  1. Data Streams received from the Service didn't work! Fixed by updating the DataStreamStorage interface to signal if the the DataStream Receiver() should be used instead of the writer.
  2. Guids could not be the return type of an RPC. Fixed by using BSON serialisation rather than JSON, BSON is what we use over the wire. Originally I was hoping to store messages in Redis with JSON to make it easier to debug, since that has never been useful and causes problems and we encrypt the data in Redis anyway switching to BSON seems reasonable.

In terms of when we test with redis:

  • We only test with Redis when testing the Latest Client with the Latest Service. In Halibut we test that the latest code works with previous versions of Halibut, those tests cases are not run with Redis since it would add no value.
  • We don't test Redis with WebSockets since the Redis queue doesn't work under net48 AND currently we can only test websockets under net48.

How to review this PR

Quality ✔️

Pre-requisites

  • I have read How we use GitHub Issues for help deciding when and where it's appropriate to make an issue.
  • I have considered informing or consulting the right people, according to the ownership map.
  • I have considered appropriate testing for my change.

var disposableCollection = new DisposableCollection();

var redisFacade = new RedisFacade("localhost:" + RedisTestHost.Port(), (Guid.NewGuid()).ToString(), log);
var redisFacade = RedisFacadeBuilder.CreateRedisFacade(port: RedisTestHost.Port());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing this means we can run the tests on a machine that doesn't have redis/docker and instead point it to another machine which does.

{
IPendingRequestQueue ?pendingRequestQueue = null;
var halibutTimeoutsAndLimits = new HalibutTimeoutsAndLimitsForTestsBuilder().Build();
halibutTimeoutsAndLimits.PollingQueueWaitTimeout = TimeSpan.FromSeconds(1);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The queues respect this timeout passed in this class, so we can just use that instead of making our own queue.



// The queues don't all work the same with the Count operator, this account for that.
int baseCount = pendingRequestQueue!.Count;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For redos the count is the number of in flight requests where for in mem queue the count is the number of yet to be collected requests.

It was hard to make the Count thing even work so that any tests could peer into the queue. This is the consequence if that.

#if NET8_0_OR_GREATER
await stream.CopyToAsync(memoryStream, ct);
#else
await stream.CopyToAsync(memoryStream);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code path is unlikely to be ever hit, and will never be hit in prod.

new CancelWhenRequestDequeuedPendingRequestQueueFactory(inner, tokenSourceToCancel, ShouldCancelOnDequeue, OnResponseApplied)))
.WithPendingRequestQueueFactoryBuilder(builder => builder.WithDecorator((_, inner) =>
{
#if NET8_0_OR_GREATER
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redis queue does not exist for net48

, leaveOpen: true
#endif
))
using (var bsonDataWriter = new BsonDataWriter(stream) { CloseOutput = false })
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We must use BSON to support services that return a guid type.

@LukeButters LukeButters changed the title Test with redis queue By default all Halibut tests will run with the Redis queue Oct 8, 2025
@LukeButters LukeButters marked this pull request as ready for review October 8, 2025 21:19
@LukeButters LukeButters requested a review from a team as a code owner October 8, 2025 21:19

[Test]
[LatestClientAndLatestServiceTestCases(testNetworkConditions: false, testListening: false, testWebSocket: false)]
[LatestClientAndLatestServiceTestCases(testNetworkConditions: false, testListening: false, testWebSocket: false, pollingQueuesToTest: PollingQueuesToTest.RedisOnly)]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are Redis specific tests so we only want redis.

}

[Test]
[LatestClientAndLatestServiceTestCases(testNetworkConditions: false, testListening: false, testWebSocket: false, pollingQueuesToTest: PollingQueuesToTest.RedisOnly)]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really this test should have existed before hand, but since I added some code around this I added a test to check it.


namespace Halibut.Tests.Queue.Redis.Utils
{
public class WithDisposablesDataStreamStorage : IStoreDataStreamsForDistributedQueues
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to copy the in memory data stream storage to be able to get something to test that disposables are indeed called.

@LukeButters LukeButters requested a review from Copilot October 8, 2025 21:24
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces the Redis queue as the default test configuration for Halibut polling tests, adding a new queue testing dimension to the test framework. The changes enable tests to run with both in-memory and Redis queues when testing the latest client and service versions.

Key changes:

  • Added infrastructure to test polling with Redis queues by default alongside in-memory queues
  • Fixed two Redis queue bugs: DataStream handling and GUID serialization by switching from JSON to BSON
  • Enhanced test framework to support different queue types with proper configuration

Reviewed Changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
PendingRequestQueueFactoryExtensionMethods.cs New extension methods for modifying and capturing Redis queue creation
RehydrateWithProgressReporting.cs Added support for using DataStream receiver vs writer based on useReceiver flag
MessageSerialiserAndDataStreamStorage.cs Updated to pass useReceiver parameter to distinguish request vs response handling
IRehydrateDataStream.cs Added DataStreamRehydrationDataDataStreamReceiver class for receiver-based rehydration
IStoreDataStreamsForDistributedQueues.cs Added useReceiver parameter to StoreDataStreams method
QueueMessageSerializer.cs Switched from JSON to BSON serialization for Redis queue compatibility
Test attribute files Enhanced test case generation to support different polling queue types
Test builder files Updated to support queue type selection and configuration

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines 24 to 29
/// <param name="useReciever">When set 'true' the data must be read from the Receiver of the DataStream. When 'false' use the WriteData() method.
/// This will be true for responses and false for Requests.</param>
/// <param name="cancellationToken"></param>
/// <returns>A string, DataStreamMetadata, containing a small amount of data that will be stored in redis, this will be
/// given to RehydrateDataStreams</returns>
public Task<byte[]> StoreDataStreams(IReadOnlyList<DataStream> dataStreams, CancellationToken cancellationToken);
public Task<byte[]> StoreDataStreams(IReadOnlyList<DataStream> dataStreams, bool useReciever, CancellationToken cancellationToken);
Copy link
Preview

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected spelling of 'useReciever' to 'useReceiver'.

Copilot uses AI. Check for mistakes.

@LukeButters LukeButters requested a review from Copilot October 8, 2025 21:27
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated 3 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@@ -0,0 +1 @@
.git/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol, I suppose this is logical. I'm surprised this was needed now, where it wasn't needed before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The profanity filter got updated by someone on the internet to include the word dumb. But git makes a file with the word dumb in it when you check out the directory.

It is somewhat of a cock up by that author, to not verify that the tool used by maaaaaany devs doesn't use the word dumb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants