Transactional Parallel Consumer stuck while rebalancing

[mikhelef]

Hi,
When starting multiple instances of parallelconsumer and having an input topic full of records, the first instance starts consuming and processsing records and then get stuck while rebalancing (when the other instances join the group) because it cannot produce the remaining records to the destination topic as it cannot get the produce lock (maybe because it is locked by the flush when partition is revoked).
After max.poll.interval.ms (6mn) the consumer is considered dead and exit the consumer group.

the used configuration is:

• PC version: 0.5.2.4
• commitMode = PERIODIC_TRANSACTIONAL_PRODUCER
• producelock = 2mn
• max.poll.records = 48
• max.poll.interval.ms = 6mn
• PC concurrency per instance (threads) = 16
• Number of partitions = 4
• Number of intances / kafka consumers = 4
• max.request.size: 30000000
• batch.size: 250000
• linger.ms: 65

I have tried increasing producelock timeout to 4 minutes but got the same result, one or multiple consumers stuck at rebalancing then got excluded from the consumer group.

Has anyone experienced this ?

Thank you.