org.springframework.kafka.support.Acknowledgment Java Examples

The following examples show how to use org.springframework.kafka.support.Acknowledgment. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: EventDispatcher.java    From spring-rdbms-cdc-kafka-elasticsearch with Apache License 2.0 6 votes vote down vote up
@KafkaListener(topics = {
    "mysqlcdc.cdc.USERS",
    "mysqlcdc.cdc.JOBS",
    "mysqlcdc.cdc.ADDRESSES"})
@Timed
public void handleEvents(List<ConsumerRecord<String, DebeziumEvent>> records,
                            Acknowledgment acknowledgment) {

  LOGGER.debug("Request to process {} records", records.size());

  List<ConsumerRecord<String, DebeziumEvent>> sortedRecords = records.stream()
      .sorted(Comparator.comparing(r -> r.value().getPayload().getDate()))
      .collect(Collectors.toList());

  sortedRecords.forEach(record -> {

    LOGGER.debug("Request to handle {} event in the topic {}", record.value().getPayload().getOperation(), record.topic());

    handlerFactory.getHandler(record.topic()).process(record.value());

  });

  acknowledgment.acknowledge();
}
 
Example #2
Source File: KafkaMetricReceiver.java    From super-cloudops with Apache License 2.0 6 votes vote down vote up
/**
 * Receiving consumer messages on multiple topics
 *
 * @param records
 * @param ack
 */
@KafkaListener(topicPattern = TOPIC_KAFKA_RECEIVE_PATTERN, containerFactory = BEAN_KAFKA_BATCH_FACTORY)
public void onMetricReceive(List<ConsumerRecord<byte[], Bytes>> records, Acknowledgment ack) {
	try {
		if (log.isDebugEnabled()) {
			log.debug("Receive metric records - {}", records);
		}
		if (log.isInfoEnabled()) {
			log.info("Receive metric records size - {}", records.size());
		}

		doProcess(records, new MultiAcknowledgmentState(ack));
	} catch (Exception e) {
		log.error(String.format("Failed to receive process for ", records.size()), e);
	}
}
 
Example #3
Source File: TorrentStoreServiceApplication.java    From Dodder with MIT License 5 votes vote down vote up
@StreamListener("index-message-in")
public void indexTorrent(Message<Torrent> message) {
	try {
		Acknowledgment acknowledgment = message.getHeaders().get(KafkaHeaders.ACKNOWLEDGMENT, Acknowledgment.class);
		Torrent torrent = message.getPayload();
		log.debug("Index torrent to elasticsearch, info hash is {}", torrent.getInfoHash());
		torrentService.index(torrent);
		//no error, execute acknowledge
		if (acknowledgment != null) {
			acknowledgment.acknowledge();
		}
	} catch (Exception e) {
		log.error("Index torrent error: {}", e);
	}
}
 
Example #4
Source File: KafkaPublishableExceptionListener.java    From enode with MIT License 5 votes vote down vote up
/**
 * Invoked with data from kafka.
 *
 * @param data           the data to be processed.
 * @param acknowledgment the acknowledgment.
 */
@Override
public void onMessage(ConsumerRecord data, Acknowledgment acknowledgment) {
    QueueMessage queueMessage = KafkaTool.covertToQueueMessage(data);
    publishableExceptionListener.handle(queueMessage, context -> {
        if (acknowledgment != null) {
            acknowledgment.acknowledge();
        }
    });
}
 
Example #5
Source File: KafkaApplicationMessageListener.java    From enode with MIT License 5 votes vote down vote up
/**
 * Invoked with data from kafka.
 *
 * @param data           the data to be processed.
 * @param acknowledgment the acknowledgment.
 */
@Override
public void onMessage(ConsumerRecord<String, String> data, Acknowledgment acknowledgment) {
    QueueMessage queueMessage = KafkaTool.covertToQueueMessage(data);
    applicationMessageListener.handle(queueMessage, context -> {
        if (acknowledgment != null) {
            acknowledgment.acknowledge();
        }
    });
}
 
Example #6
Source File: KafkaCommandListener.java    From enode with MIT License 5 votes vote down vote up
/**
 * Invoked with data from kafka.
 *
 * @param data           the data to be processed.
 * @param acknowledgment the acknowledgment.
 */
@Override
public void onMessage(ConsumerRecord<String, String> data, Acknowledgment acknowledgment) {
    QueueMessage queueMessage = KafkaTool.covertToQueueMessage(data);
    commandListener.handle(queueMessage, context -> {
        if (acknowledgment != null) {
            acknowledgment.acknowledge();
        }
    });
}
 
Example #7
Source File: KafkaDomainEventListener.java    From enode with MIT License 5 votes vote down vote up
/**
 * Invoked with data from kafka. The default implementation throws
 * {@link UnsupportedOperationException}.
 *
 * @param data           the data to be processed.
 * @param acknowledgment the acknowledgment.
 */
@Override
public void onMessage(ConsumerRecord<String, String> data, Acknowledgment acknowledgment) {
    QueueMessage queueMessage = KafkaTool.covertToQueueMessage(data);
    domainEventListener.handle(queueMessage, context -> {
        if (acknowledgment != null) {
            acknowledgment.acknowledge();
        }
    });
}
 
Example #8
Source File: KafKaRegisterSuccMailConsumer.java    From gpmall with Apache License 2.0 5 votes vote down vote up
/**
 * 指定消费某一个分区
 * @KafkaListener(id = "",topicPartitions ={@TopicPartition(topic=topic,partitions = {"1"})},containerFactory = "userRegisterSuccKafkaListenerContainerFactory",groupId = group_id)
 */
@KafkaListener(id = "",topics = topic,containerFactory = "userRegisterSuccKafkaListenerContainerFactory",groupId = group_id)
public void receiveInfo(Map userVerifyMap, Acknowledgment acknowledgment){
    try {
        log.info("收到一条注册消息"+userVerifyMap);
        sendMail(userVerifyMap);
        acknowledgment.acknowledge();//手动提交消息
    }catch (Exception e){
        e.printStackTrace();
    }
}
 
Example #9
Source File: TorrentStoreServiceApplication.java    From Dodder with MIT License 5 votes vote down vote up
@StreamListener("torrent-message-in")
public void handleTorrent(Message<Torrent> message) {
	try {
		Acknowledgment acknowledgment = message.getHeaders().get(KafkaHeaders.ACKNOWLEDGMENT, Acknowledgment.class);
		Torrent torrent = message.getPayload();
		log.debug("Save torrent to MongoDB, info hash is {}", torrent.getInfoHash());
		torrentService.upsert(torrent);
		//no error, execute acknowledge
		if (acknowledgment != null) {
			acknowledgment.acknowledge();
		}
	} catch (Exception e) {
		log.error("Insert or update torrent error: {}", e);
	}
}
 
Example #10
Source File: AbstractKafkaConsumer.java    From integration-patterns with MIT License 5 votes vote down vote up
protected void handleConsumerRecord(final ConsumerRecord<String, String> consumerRecord, final Acknowledgment ack) {
    LOG.info("Received {}", ConsumerRecordLoggingHelper.toLogSafeString(consumerRecord, payloadSensitive));
    final EventProcessingState state = processAndMapExceptionsToState(consumerRecord);
    if (EventProcessingState.UNEXPECTED_ERROR == state) {
        unprocessableEventService.save(new UnprocessedEventEntity(consumerRecord));
    } else if (EventProcessingState.TEMPORARY_ERROR == state) {
        throw new TemporaryKafkaProcessingError("Message processing failed temporarily");
    }
    ack.acknowledge();
}
 
Example #11
Source File: MessageHandler.java    From spring-boot-demo with MIT License 5 votes vote down vote up
@KafkaListener(topics = KafkaConsts.TOPIC_TEST, containerFactory = "ackContainerFactory")
public void handleMessage(ConsumerRecord record, Acknowledgment acknowledgment) {
    try {
        String message = (String) record.value();
        log.info("收到消息: {}", message);
    } catch (Exception e) {
        log.error(e.getMessage(), e);
    } finally {
        // 手动提交 offset
        acknowledgment.acknowledge();
    }
}
 
Example #12
Source File: StubRunnerKafkaRouter.java    From spring-cloud-contract with Apache License 2.0 4 votes vote down vote up
@Override
public void onMessage(ConsumerRecord<Object, Object> data,
		Acknowledgment acknowledgment, Consumer<?, ?> consumer) {
	onMessage(data);
}
 
Example #13
Source File: StubRunnerKafkaRouter.java    From spring-cloud-contract with Apache License 2.0 4 votes vote down vote up
@Override
public void onMessage(ConsumerRecord<Object, Object> data,
		Acknowledgment acknowledgment) {
	onMessage(data);
}
 
Example #14
Source File: S1pKafkaApplication.java    From grussell-spring-kafka with Apache License 2.0 4 votes vote down vote up
@Override
public void onMessage(ConsumerRecord<String, String> record, Acknowledgment acknowledgment) {
	System.out.println("Received: " + record.value());
	acknowledgment.acknowledge();
	this.latch.countDown();
}
 
Example #15
Source File: KafkaBinderTests.java    From spring-cloud-stream-binder-kafka with Apache License 2.0 4 votes vote down vote up
@Test
@SuppressWarnings("unchecked")
public void testManualAckSucceedsWhenAutoCommitOffsetIsTurnedOff() throws Exception {
	Binder binder = getBinder();

	DirectChannel moduleOutputChannel = createBindableChannel("output",
			createProducerBindingProperties(createProducerProperties()));
	QueueChannel moduleInputChannel = new QueueChannel();

	Binding<MessageChannel> producerBinding = binder.bindProducer(
			"testManualAckSucceedsWhenAutoCommitOffsetIsTurnedOff",
			moduleOutputChannel, createProducerProperties());

	ExtendedConsumerProperties<KafkaConsumerProperties> consumerProperties = createConsumerProperties();
	consumerProperties.getExtension().setAutoCommitOffset(false);

	Binding<MessageChannel> consumerBinding = binder.bindConsumer(
			"testManualAckSucceedsWhenAutoCommitOffsetIsTurnedOff", "test",
			moduleInputChannel, consumerProperties);

	String testPayload1 = "foo" + UUID.randomUUID().toString();
	Message<?> message1 = org.springframework.integration.support.MessageBuilder
			.withPayload(testPayload1.getBytes()).build();

	// Let the consumer actually bind to the producer before sending a msg
	binderBindUnbindLatency();
	moduleOutputChannel.send(message1);

	Message<?> receivedMessage = receive(moduleInputChannel);
	assertThat(receivedMessage).isNotNull();
	assertThat(receivedMessage.getHeaders().get(KafkaHeaders.ACKNOWLEDGMENT))
			.isNotNull();
	Acknowledgment acknowledgment = receivedMessage.getHeaders()
			.get(KafkaHeaders.ACKNOWLEDGMENT, Acknowledgment.class);
	try {
		acknowledgment.acknowledge();
	}
	catch (Exception e) {
		fail("Acknowledge must not throw an exception");
	}
	finally {
		producerBinding.unbind();
		consumerBinding.unbind();
	}
}
 
Example #16
Source File: KafkaSampleConsumer.java    From kafka-service-broker with Apache License 2.0 4 votes vote down vote up
@Override
public void onMessage(ConsumerRecord<Integer, String> data, Acknowledgment acknowledgment) {
    log.info("received: " + data);
}
 
Example #17
Source File: OrderKafkaListener.java    From microservice-kafka with Apache License 2.0 4 votes vote down vote up
@KafkaListener(topics = "order")
public void order(Invoice invoice, Acknowledgment acknowledgment) {
	log.info("Received invoice " + invoice.getId());
	invoiceService.generateInvoice(invoice);
	acknowledgment.acknowledge();
}
 
Example #18
Source File: OrderKafkaListener.java    From microservice-kafka with Apache License 2.0 4 votes vote down vote up
@KafkaListener(topics = "order")
public void order(Shipment shipment, Acknowledgment acknowledgment) {
	log.info("Received shipment " + shipment.getId());
	shipmentService.ship(shipment);
	acknowledgment.acknowledge();
}
 
Example #19
Source File: ProductEventConsumer.java    From integration-patterns with MIT License 4 votes vote down vote up
@KafkaListener(topics = "${eventing.topic_name}")
public void listen(final ConsumerRecord<String, String> consumerRecord, final Acknowledgment ack) {
    super.handleConsumerRecord(consumerRecord, ack);
}
 
Example #20
Source File: KafkaMetricReceiver.java    From super-cloudops with Apache License 2.0 4 votes vote down vote up
public MultiAcknowledgmentState(Acknowledgment ack) {
	super();
	this.ack = ack;
}
 
Example #21
Source File: KafkaEventArgListener.java    From bird-java with MIT License 3 votes vote down vote up
/**
 * 消费者拿到消息时就提交offset,消费失败时在消费者内部重试
 * 避免消费者服务中多个事件处理程序情况下,一个处理程序失败,导致其他处理程序重试的问题
 *
 * @param data
 * @param acknowledgment
 */
@Override
public void onMessage(ConsumerRecord<String, EventArg> data, Acknowledgment acknowledgment) {
    if (data == null) return;
    dispatcher.enqueue(data.value());
    acknowledgment.acknowledge();
}