Tracking: PostgreSQL support in KFP

[zijianjoy]

PostgreSQL request has become the top upvoted issue on KFP repo: #7512. This issue is for tracking the work of this integration.

• KFP Backend integration

  • Define the DB config for PostgreSQL DB info
    • Basic connection config
    • Secure connection config: passfile, hostaddr, ssl mode, certificate, etc.
    • Flag to switch between Postgresql driver and MySQL driver
  • Upgrade GORM to v2 #9859
  • Syntax support of PostgreSQL DB in backend
    • Adopt a different syntax during initialization. (schema difference)
    • Adopt a different syntax during execution. (Modification of data)
    • Explore the dialect difference and develop a control mechanism to enable easy testing on the storage layer.
  • Testing
    • Unit testing for Postgresql behavior.
    • Functional testing for Postgresql behavior.
    • E2E testing for Postgresql behavior.
  • Cache server integration
  • KFP API server integration
  • Manifest support for Postgresql
    • Standalone KFP manifests for postgres #9860
    • Full Kubeflow manifests for postgres #9861
    • Marketplace KFP (managed CloudSQL instance)

• MLMD integration

  • Develop Postgresql integration on MLMD
    • Investigate String property max chars when using MLMD with PostgreSQL google/ml-metadata#194 (comment)
  • Release new version of MLMD
  • Upgrade KFP to newer version of MLMD #9848
  • Manifest support for Postgresql in MLMD
    • Standalone KFP
    • Full Kubeflow Kubeflow
    • Marketplace KFP (managed CloudSQL instance)

[zijianjoy]

cc @chensun

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.