feat(policy)!: Add manager column to provider configuration for multi-instance support #2601

strantalis · 2025-07-30T17:49:17Z

Proposed Changes

This PR implements the provider configuration manager column enhancement and includes a significant architecture improvement to eliminate code duplication.

Core Features

✅ Added manager column to provider_config table with composite unique constraint (provider_name + manager)
✅ Database migration with backward compatibility (existingrecords default to 'local' manager)
✅ Updated protobuf definitions to require manager field in API requests
✅ Enhanced SQL queries and audit logging to support manager field

Key Benefits

Multiple Instance Support: Create multiple provider configs with same name but different manager types
No Code Duplication: Trust system is single source of truth for available managers
Automatic Discovery: Well-known configuration endpoint automatically exposes registered trust key managers
Premium Integration: Custom managers integrate seamlessly via existing WithTrustKeyManagerFactories pattern
Backward Compatibility: Existing providers automatically assigned 'local' manager type

Technical Changes

Database: Added manager column with composite unique constraint
API: Manager field now required in CreateProviderConfig, optional in UpdateProviderConfig
Service: Validates against RegistrationParams.KeyManagerFactories instead of separate registry
Well-known Config: Dynamically populated from trust key manager factories
Tests: Updated to reflect new validation approach

Usage for Custom Managers

Premium builds can add custom managers using the existing
pattern:

  server.Start(
      server.WithTrustKeyManagerFactories(
          trust.NamedKeyManagerFactory{Name: "premium-hsm",
  Factory: hsmFactory},
          trust.NamedKeyManagerFactory{Name: "premium-vault",
  Factory: vaultFactory},
      ),
  )

Checklist

I have added or updated unit tests
I have added or updated integration tests (if appropriate)
I have added or updated documentation

Testing Instructions

gemini-code-assist

Summary of Changes

Hello @strantalis, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the key management service by introducing multi-instance support for key provider configurations. It achieves this by adding a new manager field to the provider_config entity, allowing for distinct configurations of the same provider type under different management contexts. This change includes comprehensive updates to the database schema, API definitions, and backend logic, ensuring robust validation and backward compatibility.

Highlights

Multi-Instance Key Provider Support: Introduced a manager column to the provider_config table, enabling multiple key provider configurations to share the same provider_name if they belong to different manager types. This supports diverse key management deployment scenarios.
Database Schema Evolution: A new migration script adds the manager column, defaults existing records to 'local' for backward compatibility, and establishes a composite unique constraint on (provider_name, manager).
API and Protocol Updates: The manager field has been integrated into CreateProviderConfig, GetProviderConfig, and UpdateProviderConfig requests and responses across protobuf definitions, gRPC documentation, and Go service implementations.
Manager Type Validation and Registration: A system for registering and validating manager types has been implemented. New CreateProviderConfig and UpdateProviderConfig requests now validate that the provided manager type is registered, preventing invalid configurations. Registered manager types are also exposed via the well-known configuration endpoint.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

New column appears, Managers now guide the keys, Multi-instance blooms.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces the manager column to the provider_config table, enabling multi-instance support for provider configurations. The changes are well-integrated across the database, service logic, and API definitions. However, there are potential issues with the database migration rollback and a breaking change in the public API that need to be addressed.

service/policy/db/migrations/20250730000000_add_provider_manager_column.sql

service/policy/keymanagement/key_management.proto

service/policy/db/models.go

service/policy/keymanagement/key_management.go

protocol/go/CHANGELOG.md

service/wellknownconfiguration/wellknown_configuration.go

dmihalcik-virtru · 2025-07-31T18:55:41Z

service/policy/keymanagement/key_management.proto

@@ -32,6 +35,8 @@ message GetProviderConfigRequest {
    string id = 2 [(buf.validate.field).string.uuid = true];
    string name = 3 [(buf.validate.field).string.min_len = 1];
  }
+  // Optional - filter by manager type when searching by name
+  string manager = 4 [(buf.validate.field).required = false];


Should we perhaps add a ListProviderConfig* operation for this, instead of making this return a value or not?

a list providers config request already exists

platform/service/policy/keymanagement/key_management.proto

Lines 40 to 48 in 3bb8f5c

message ListProviderConfigsRequest {

// Optional

policy.PageRequest pagination = 10;

}

message ListProviderConfigsResponse {

repeated KeyProviderConfig provider_configs = 1;

policy.PageResponse pagination = 10;

}

service/integration/keymanagement_test.go

service/policy/db/models.go

### Proposed Changes These changes originally showed up in #2601. ### Checklist - [ ] I have added or updated unit tests - [ ] I have added or updated integration tests (if appropriate) - [ ] I have added or updated documentation ### Testing Instructions

multi-instance support - Add `manager` column to provider_config table with composite unique constraint - Implement manager type registration system with default types (local, aws, gcp, azure, hashicorp-vault, hsm) - Update API to require manager field in CreateProviderConfig requests - Add manager validation to reject unregistered manager types - Integrate manager registry with well-known configuration endpoint - Ensure backward compatibility by defaulting existing records to 'local' manager - Update all SQL queries and protobuf definitions to support manager field - Add comprehensive test coverage for manager registry and validation logic This enables multiple provider configurations with the same name but different manager types, supporting diverse key management deployment scenarios while maintaining API compatibility.

github-actions bot added comp:db DB component comp:policy Policy Configuration ( attributes, subject mappings, resource mappings, kas registry) docs Documentation size/m labels Jul 30, 2025

gemini-code-assist bot reviewed Jul 30, 2025

View reviewed changes

strantalis marked this pull request as ready for review July 31, 2025 14:30

strantalis requested review from a team as code owners July 31, 2025 14:30

c-r33d reviewed Jul 31, 2025

View reviewed changes

protocol/go/CHANGELOG.md Outdated Show resolved Hide resolved

dmihalcik-virtru requested changes Jul 31, 2025

View reviewed changes

strantalis changed the title ~~feat(policy): Add manager column to provider configuration for multi-instance support~~ feat(policy)!: Add manager column to provider configuration for multi-instance support Aug 1, 2025

strantalis mentioned this pull request Aug 1, 2025

fix: updated generated sqlc #2609

Merged

3 tasks

strantalis added 14 commits August 28, 2025 15:15

simplify

d133669

don't introduce prot breaking changes

dbf4cb9

fix lint issues

d2c5e02

minor fixes

d9a96e0

fix proto breaking changes

6f4f211

integration tests

ddea664

minor fixes

55cd41b

fix lint issues

7440489

fix wellknown config

5e4ccd8

wellknown configuration endpoint fixes

0e7573a

proto changes

b22dc7d

update key_indexer to return proper manager name

1c29a90

register basic key manager to keymanagement

b98604a

strantalis added 2 commits August 28, 2025 15:16

don't add opentdf.io/basic to wellknown

58c021f

revert wellknown changes and refactor local var into SetupTest func

bb8f270

strantalis force-pushed the dspx-1471/provider-manager branch from e802456 to bb8f270 Compare August 28, 2025 19:16

strantalis added 2 commits August 28, 2025 15:18

sync db models

c3d2331

fix lint issues

af07b01

strantalis requested review from jakedoublev, dmihalcik-virtru, c-r33d and jrschumacher August 28, 2025 22:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(policy)!: Add manager column to provider configuration for multi-instance support #2601

feat(policy)!: Add manager column to provider configuration for multi-instance support #2601

Uh oh!

strantalis commented Jul 30, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dmihalcik-virtru Jul 31, 2025

Uh oh!

strantalis Aug 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

	message ListProviderConfigsRequest {
	// Optional
	policy.PageRequest pagination = 10;
	}
	message ListProviderConfigsResponse {
	repeated KeyProviderConfig provider_configs = 1;

	policy.PageResponse pagination = 10;
	}

feat(policy)!: Add manager column to provider configuration for multi-instance support #2601

Are you sure you want to change the base?

feat(policy)!: Add manager column to provider configuration for multi-instance support #2601

Uh oh!

Conversation

strantalis commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed Changes

Checklist

Testing Instructions

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dmihalcik-virtru Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

strantalis Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

strantalis commented Jul 30, 2025 •

edited

Loading