Random issue happen with Testcontainers.BigQuery when run on GitHub CI

[mohamedrabie]

Testcontainers.BigQuery version
4.3.0

Using the latest Testcontainers version?
Yes

.NET version
8.0.100

DockerInfo:

Connected to Docker:
  Host: unix:///var/run/docker.sock
  Server Version: 24.0.7
  Kernel Version: 5.14.0-427.50.1.el9_4.x86_64
  API Version: 1.43
  Operating System: Ubuntu 22.04.5 LTS (containerized)
  Total Memory: 377.89 GB

What happened?
Trying to create dataset in BigQuery
Example code:

public class RepositoryTests : IAsyncLifetime
{

    private readonly Mock<IOptions<GoogleCloudConfiguration>> _optionsMock;
    private readonly BigQueryContainer _bigQueryContainer;
    public RepositoryTests(ITestOutputHelper testOutputHelper)
    {
        var autoMocker = new AutoMocker();

        var googleCloudConfig = new GoogleCloudConfiguration
        {
            ProjectId = "test_project",
            DatasetId = "test_dataset",
            TableId = "test_table"
        };

        _optionsMock = autoMocker.GetMock<IOptions<GoogleCloudConfiguration>>();
        _optionsMock.Setup(o => o.Value).Returns(googleCloudConfig);

        var webHostEnvironmentMock = autoMocker.GetMock<IWebHostEnvironment>();
        webHostEnvironmentMock.Setup(env => env.EnvironmentName).Returns("local");

        var logger = XUnitLogger.CreateLogger<AuditRepositoryTests>(testOutputHelper);

        Environment.SetEnvironmentVariable("TESTCONTAINERS_RYUK_CONTAINER_IMAGE", "ARTIFACTORY_HOSTNAME/testcontainers/ryuk:0.5.1");

        _bigQueryContainer = new BigQueryBuilder()
            .WithImage("ARTIFACTORY_HOSTNAME/goccy/bigquery-emulator:0.4")
            .WithProject(googleCloudConfig.ProjectId)
            .WithCommand("--log-level", "debug")
            .WithLogger(logger)
            .Build();
    }

    [Fact]
    public async Task BigQueryIntegration_CreateAuditLog_ShouldInsertDataCorrectly()
    {
        // Arrange
        var googleCloudConfiguration = _optionsMock.Object.Value;

        using var bigQueryClient = await new BigQueryClientBuilder
        {
            BaseUri = _bigQueryContainer.GetEmulatorEndpoint(),
            ProjectId = googleCloudConfiguration.ProjectId,
            Credential = new FakeCredential()
        }
        .BuildAsync()
        .ConfigureAwait(true);

        var dataset = await bigQueryClient.CreateDatasetAsync(googleCloudConfiguration.DatasetId).ConfigureAwait(true);

    }
    public async Task InitializeAsync()
    {
        await _bigQueryContainer.StartAsync().ConfigureAwait(false);
    }

    public async Task DisposeAsync()
    {
        await _bigQueryContainer.DisposeAsync().AsTask();
    }
}

internal sealed class FakeCredential : ICredential
{
    public void Initialize(ConfigurableHttpClient httpClient)
    {
    }

    public Task<string> GetAccessTokenForRequestAsync(string authUri, CancellationToken cancellationToken)
    {
        return Task.FromResult(string.Empty);
    }
}

Relevant log output

 Error Message:
   Google.GoogleApiException : The service bigquery has thrown an exception. HttpStatusCode is Forbidden. No error message was specified.
---- Newtonsoft.Json.JsonReaderException : Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
  Stack Trace:
     at Google.Apis.Responses.HttpResponseMessageExtensions.DeserializeErrorAsync(HttpResponseMessage response, String name, ISerializer serializer)
   at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response)
   at Google.Apis.Requests.ClientServiceRequest`1.ExecuteAsync(CancellationToken cancellationToken)
   at Google.Cloud.BigQuery.V2.BigQueryClientImpl.CreateDatasetAsync(DatasetReference datasetReference, Dataset resource, CreateDatasetOptions options, CancellationToken cancellationToken)

Pulling from testcontainers/ryuk
Pulling fs layer
Pulling fs layer
Pulling fs layer
Downloading
[==================================================>]  
Verifying Checksum
Download complete
Verifying Checksum
Download complete
Verifying Checksum
Download complete
Extracting
[==================================================>]  
Pull complete
Digest: sha256:533abc56c07b52a26c955d1e7ae428d810582ab01c156384ae79960eb5fa0775
Status: Downloaded newer image for ARTIFACTORY_HOSTNAME/testcontainers/ryuk:0.5.1
Docker image ARTIFACTORY_HOSTNAME/testcontainers/ryuk:0.5.1 created
Docker container 2568ff99ec1b created
Start Docker container 2568ff99ec1b
Wait for Docker container 2568ff99ec1b to complete readiness checks
Docker container 2568ff99ec1b ready
Pulling from goccy/bigquery-emulator
Pulling fs layer
Pulling fs layer
Pulling fs layer
Download complete

Downloading
[===========================================>       ] 
Verifying Checksum
Download complete

[=================================================> ] 
Extracting
[==================================================>]  5
Pull complete
Extracting
[==================================================>]  
Pull complete
Extracting
[==================================================>]  
Pull complete
Digest: sha256:967b9efed8c5d0228b8f0370ce37c387df8a799ad53a34c2e96f6a8dfcb0dda2
Status: Downloaded newer image for ARTIFACTORY_HOSTNAME/goccy/bigquery-emulator:0.4
Docker image ARTIFACTORY_HOSTNAME/goccy/bigquery-emulator:0.4 created
Docker container b2e7e5809c30 created
Start Docker container b2e7e5809c30
Wait for Docker container b2e7e5809c30 to complete readiness checks
Docker container b2e7e5809c30 ready

Stdout: 

2025-03-28T15:10:05.603177344Z [bigquery-emulator] REST server listening at 0.0.0.0:9050
2025-03-28T15:10:05.603208380Z [bigquery-emulator] gRPC server listening at 0.0.0.0:9060

Stderr: 
Delete Docker container b2e7e5809c30

[HofmeisterAn]

Based on experiences with other modules, this error message sounds like that the service running inside the container is not fully up and running. This typically happens when the wait strategy indicates readiness too early. For this module, we wait until the message (?s).*listening.*$ appears in the output. I think it makes sense to reach out to the upstream repository for guidance, as I don't believe there's much more we can do.