Enable deploy Kafka Connect in distributed mode

[rmarting]

SUMMARY

Kafka Connect cluster can be deployed in two different modes:

• standalone, running on a single node. This is the current implementation of the amq_streams_connect role.
• distributed, running accross more than one worker nodes.

The standalone mode is intended for development, testing or small production environments. However, the distributed mode is more suitable for production environments, distributing the workloads among them.

It should be a good feature allowing to deploy a Kafka Connect cluster using the distributed mode. It should required to:

• Include the connect-distributed.properties file to set up the Kafka Cluster
• Identify the boostrap server connection, normally based in a list of different brokers

References:

• Using Kafka Connect in distributed mode

BONUS TRACK

The integration against the Kafka cluster (boostrap server connection string) should cover the different authentication mechanisms allowed by the amq_streams_broker role (e.g: SASL PLAIN, SASL SCRAM, ...)

ISSUE TYPE

• Feature Idea