added options for maxCommitDelay and maxNumRows

[sm745052]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @sm745052, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces two new optional parameters, maxCommitDelay and maxNumRows, to the SourceDbToSpanner pipeline. These parameters provide greater control over how data is written to Google Cloud Spanner, enabling users to fine-tune write operations for improved throughput and transaction management. The changes involve updating the pipeline options, modifying the Spanner configuration logic, and enhancing the Spanner writer component to incorporate these new settings.

Highlights

• Spanner Write Optimization Options: Introduced maxCommitDelay and maxNumRows as new template parameters for the SourceDbToSpanner pipeline, allowing for fine-grained control over Spanner write operations.
• maxCommitDelay Implementation: The maxCommitDelay parameter, when set to a non-negative value, is now used to configure the Spanner client for optimizing write throughput, as per the referenced Spanner documentation.
• maxNumRows Implementation: The maxNumRows parameter allows controlling the maximum number of row updates within a single Spanner transaction, enhancing batching flexibility and performance.
• SpannerWriter Updates: The SpannerWriter class has been updated to accept and apply the maxNumRows setting to the SpannerIO.Write operation, ensuring these new parameters are utilized during data writes.
• Test Coverage: Added comprehensive unit tests to verify the correct handling and application of both maxCommitDelay and maxNumRows parameters within the pipeline and writer components.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[codecov]

Codecov Report

❌ Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 55.35%. Comparing base (6381f56) to head (03f9f91).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...d/teleport/v2/templates/MigrateTableTransform.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2918      +/-   ##
============================================
+ Coverage     50.06%   55.35%   +5.29%     
+ Complexity     4972     1584    -3388     
============================================
  Files           967      462     -505     
  Lines         59329    25916   -33413     
  Branches       6448     2706    -3742     
============================================
- Hits          29702    14347   -15355     
+ Misses        27522    10701   -16821     
+ Partials       2105      868    -1237     

Components	Coverage Δ
spanner-templates	71.38% <95.45%> (+0.89%)	⬆️
spanner-import-export	∅ <ø> (∅)
spanner-live-forward-migration	79.69% <ø> (ø)
spanner-live-reverse-replication	77.42% <ø> (-0.01%)	⬇️
spanner-bulk-migration	88.23% <95.45%> (+0.02%)	⬆️

Files with missing lines	Coverage Δ
...cloud/teleport/v2/templates/SourceDbToSpanner.java	30.00% <100.00%> (+7.77%)	⬆️
...google/cloud/teleport/v2/writer/SpannerWriter.java	78.12% <100.00%> (+13.12%)	⬆️
...d/teleport/v2/templates/MigrateTableTransform.java	0.00% <0.00%> (ø)

... and 527 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.