org.apache.kafka.common.record.InvalidRecordException Java Examples

The following examples show how to use org.apache.kafka.common.record.InvalidRecordException. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.

Example #1

Source File: CustomPartitioner.java From ad with Apache License 2.0

6 votes

/**
 * 决定消息被写入哪个分区
 * @param topic topic
 * @param key key
 * @param keyBytes key serialize byte
 * @param value value
 * @param valueBytes value serialize byte
 * @param cluster kakfa cluster
 * @return
 */
@Override
public int partition(String topic, Object key, byte[] keyBytes,
                     Object value, byte[] valueBytes, Cluster cluster) {
    // 所有分区信息
    List<PartitionInfo> partitionInfos = cluster.partitionsForTopic(topic);
    int partitionCount = partitionInfos.size();
    // 要求必须存在 key,如果key 是"name" 就分配到最后一个分区, 其他key hash取模
    if (keyBytes == null || !key.getClass().equals(String.class)) {
        throw new InvalidRecordException("kafka message must have a String key");
    }
    if (partitionCount == 1 || StringUtils.endsWithIgnoreCase("name", key.toString())) {
        return partitionCount - 1;
    }
    return Math.abs(Utils.murmur2(keyBytes)) % (partitionCount - 1);
}

Example #2

Source File: Record.java From feeyo-redisproxy with BSD 3-Clause "New" or "Revised" License

5 votes

private static void readHeaders(ByteBuffer buffer, int numHeaders) {
	for (int i = 0; i < numHeaders; i++) {
		int headerKeySize = ByteUtils.readVarint(buffer);
		if (headerKeySize < 0)
			throw new InvalidRecordException("Invalid negative header key size " + headerKeySize);

		Utils.utf8(buffer, headerKeySize);
		buffer.position(buffer.position() + headerKeySize);

		int headerValueSize = ByteUtils.readVarint(buffer);
		if (headerValueSize >= 0) {
			buffer.position(buffer.position() + headerValueSize);
		}
	}
}

Example #3

Source File: OddEvenPartitioner.java From Kafka-Streams-Real-time-Stream-Processing with The Unlicense

3 votes

/**
 * Simple partitioning on the message key
 * Odd keys to partition 0
 * Even keys to partition 1
 *
 * @param topic topic Name
 * @param key Message Key
 * @param keyBytes Key Bytes
 * @param value Message Value
 * @param valueBytes Value Bytes
 * @param cluster Cluster Object
 * @return Partition Id
 */
@Override
public int partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster) {
    if ((keyBytes == null) || (!(key instanceof Integer)))
        throw new InvalidRecordException("Topic Key must have a valid Integer value.");

    if (cluster.partitionsForTopic(topic).size() != 2)
        throw new InvalidTopicException("Topic must have exactly two partitions");

    return (Integer) key % 2;
}