Committing offsets when manually committing

[pauldambra]

Hi,

We're building an application that buffers messages from kafka. To avoid getting stuck in detail here we can't process each message individually so we have turned off auto-commit. Every so often a message is received such that we know we've successfully processed up to that point and we can manually commit.

(In case it matters we're using node-rdkafka library in typescript)

This documentation https://kafka.apache.org/10/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html says

Note: The committed offset should always be the offset of the next message that your application will read. Thus, when calling commitSync(offsets) you should add one to the offset of the last message processed.

I'm worried I'm misunderstanding this.

If I have received a kafka message with offset 1234, I should commit offset 1235

This would ensure that if my consumer restarted at that point the first message it receives would be 1235

Is that right?

It seems counter-intuitive to me so I'm worried I'm misunderstanding.

(thanks in advance for your time 😊)