Spring Kafka Message Ordering

[akoufa]

Hello, when using Spring Kafka and the KafkaListener annotation with Kotlin suspend functions and asyncAcks enabled, is there any guarantee that the Kafka messages of a single partition will be consumed and processed in the order they were produced ? What if I have suspending calls in my consumption could multiple events then be forwarded to my listener with the order being then random?

[artembilan]

The suspend is turned to Mono which is subscribed from the consumer thread. So, that guarantees an order according to Kafka recommendations. The rest of async handling in the listener method is out of the framework responsibility.
Why would you go suspend if ordering is crucial for you? The different partitions are handled in their own threads.

[akoufa]

It wasn't until recent changes. We had used suspend because we have a Spring Boot Webflux application and the Webclient and other APIs we use based on the event consumption provide async APIs. What would be your suggestion? Can we use Spring Kafka with blocking approach, even in a Webflux app ? Or should we consume batches and iterate through them honoring the order ?

[artembilan]

Right. That is really the whole point of async @KafkaListener - do not care about the order.
If you care about the order, make sure you handle all the data from the same partition in the same thread.
Or might be same Sinks.many() where you feed that data into your WebClient.
Not sure how to model a suspend function for that goal, but you indeed may just start from the regular void @KafkaListener and see how your WebClient behaves.
I don't think it should report you anything since the @KafkaListener call is done from the consumer thread.
At the same time there is no guarantee that calling WebClient would give us the same order on the server side.
So, again: ordering and async logic somehow contradict each other.