在spring kafka consumer 1.2.x中寻求方法行为

时间:2018-05-10 13:41:17

标签: java apache-kafka kafka-consumer-api spring-kafka

我不想为那些处理失败的消息提交偏移量,我希望它们能够再次重新传递以进行处理。我正在使用spring-kafka 1.2.x并在我的监听器中实现了 ConsumerSeekAware

@Component
public class Listener implements ConsumerSeekAware {

    private static Logger logger = LoggerFactory.getLogger(Listener.class);

    private final ThreadLocal<ConsumerSeekCallback> seekCallBack = new ThreadLocal<>();

    @KafkaListener(topics = "my-topic", containerFactory = "kafkaManualAckListenerContainerFactory")
    public void listen1(ConsumerRecord<String, String> consumerRecord) throws MyCustomException {
        logger.info("received: key - " + consumerRecord.key() + " value - " + consumerRecord.value());

        // Below code is just to show the issue.Not acknowledging so I can get the same msg again.
        boolean should_commit = false;
        if(should_commit) {
            ack.acknowledge();
        }
        else {
            this.seekCallBack.get().seek(consumerRecord.topic(), consumerRecord.partition() , consumerRecord.offset());
        }
    }

    @Override
    public void registerSeekCallback(ConsumerSeekCallback callback) {
        logger.info("registerSeekCallback called..");
        this.seekCallBack.set(callback);
    }

    @Override
    public void onPartitionsAssigned(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback) {
     logger.info("onPartitionsAssigned called..");
    }

    @Override
    public void onIdleContainer(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback) {
        logger.info("onIdleContainer called..");
    }
}
######### Contaianer config(auto.commit在消费者中为false)
factory.getContainerProperties().setAckOnError(false);
          factory.getContainerProperties().setAckMode(AbstractMessageListenerContainer.AckMode.MANUAL_IMMEDIATE);

我面临的问题是,如果我在一个主题的不同分区中有10条消息,那么我将逐一获取所有这些消息,并在获取所有消息后继续获取任何分区的最后消息。我还尝试了在版本2.0.x中实现的 SeekToCurrentErrorHandler ,它完美无缺。但我无法升级我的卡夫卡版本。如果我重新启动容器,我会再次收到所有消息,但是我不希望在处理消息失败时停止容器。

所以我的问题是,甚至可以在spring-kafka 1.2.x中使用与 SeekToCurrentErrorHandler 相同的行为(完全相同而不需要停止容器)行为?

0 个答案:

没有答案