Skip to content

Add configurable HAProxy backend health check parameters without requiring full configuration override #2206

@timstoop

Description

@timstoop

Currently, to customize HAProxy backend health check intervals and failover behavior, users must provide a complete HAProxy configuration via haproxy.configuration, duplicating all the operator's default logic and making maintenance difficult.

Problem

When a PXC node fails (e.g., during rolling restart), HAProxy takes 20+ seconds to detect the failure with default settings:

  • check inter 10000 (10s interval)
  • fall 2 (2 failures = 20s total)

Worse, existing client connections to the failed backend are NOT terminated, causing them to hang until TCP timeout (potentially minutes), as on-marked-down shutdown-sessions is not configured.

Current Workaround

Provide full HAProxy configuration in haproxy.configuration field, but this:

  • Duplicates operator logic
  • Breaks on operator upgrades
  • Is difficult to maintain

Requested Feature

Add CR fields to configure HAProxy backend health check parameters without full config override:

haproxy:
  healthCheck:
    interval: 3000          # check inter 3000 (default: 10000)
    fall: 2                 # fall 2 (default: 2)
    rise: 1                 # rise 1 (default: 1)
    shutdownOnMarkDown: true # on-marked-down shutdown-sessions (default: false)

Benefit

  • Fast failover (6s with interval: 3000)
  • Active connection termination on backend failure
  • No maintenance burden
  • Survives operator upgrades

Alternative

If granular options are too complex, at least expose a "fast-failover" preset that applies sensible defaults.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions