org.apache.flink.runtime.state.heap.HeapPriorityQueueElement Java Examples

The following examples show how to use org.apache.flink.runtime.state.heap.HeapPriorityQueueElement. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: RocksDBCachingPriorityQueueSet.java    From Flink-CEPplus with Apache License 2.0 6 votes vote down vote up 
RocksDBCachingPriorityQueueSet(
	@Nonnegative int keyGroupId,
	@Nonnegative int keyGroupPrefixBytes,
	@Nonnull RocksDB db,
	@Nonnull ColumnFamilyHandle columnFamilyHandle,
	@Nonnull TypeSerializer<E> byteOrderProducingSerializer,
	@Nonnull DataOutputSerializer outputStream,
	@Nonnull DataInputDeserializer inputStream,
	@Nonnull RocksDBWriteBatchWrapper batchWrapper,
	@Nonnull OrderedByteArraySetCache orderedByteArraySetCache) {
	this.db = db;
	this.columnFamilyHandle = columnFamilyHandle;
	this.byteOrderProducingSerializer = byteOrderProducingSerializer;
	this.batchWrapper = batchWrapper;
	this.outputView = outputStream;
	this.inputView = inputStream;
	this.orderedCache = orderedByteArraySetCache;
	this.allElementsInCache = false;
	this.groupPrefixBytes = createKeyGroupBytes(keyGroupId, keyGroupPrefixBytes);
	this.seekHint = groupPrefixBytes;
	this.internalIndex = HeapPriorityQueueElement.NOT_CONTAINED;
}
Example #2
Source File: RocksDBCachingPriorityQueueSet.java    From flink with Apache License 2.0 6 votes vote down vote up 
RocksDBCachingPriorityQueueSet(
	@Nonnegative int keyGroupId,
	@Nonnegative int keyGroupPrefixBytes,
	@Nonnull RocksDB db,
	@Nonnull ReadOptions readOptions,
	@Nonnull ColumnFamilyHandle columnFamilyHandle,
	@Nonnull TypeSerializer<E> byteOrderProducingSerializer,
	@Nonnull DataOutputSerializer outputStream,
	@Nonnull DataInputDeserializer inputStream,
	@Nonnull RocksDBWriteBatchWrapper batchWrapper,
	@Nonnull OrderedByteArraySetCache orderedByteArraySetCache) {
	this.db = db;
	this.readOptions = readOptions;
	this.columnFamilyHandle = columnFamilyHandle;
	this.byteOrderProducingSerializer = byteOrderProducingSerializer;
	this.batchWrapper = batchWrapper;
	this.outputView = outputStream;
	this.inputView = inputStream;
	this.orderedCache = orderedByteArraySetCache;
	this.allElementsInCache = false;
	this.groupPrefixBytes = createKeyGroupBytes(keyGroupId, keyGroupPrefixBytes);
	this.seekHint = groupPrefixBytes;
	this.internalIndex = HeapPriorityQueueElement.NOT_CONTAINED;
}
Example #3
Source File: RocksDBCachingPriorityQueueSet.java    From flink with Apache License 2.0 6 votes vote down vote up 
RocksDBCachingPriorityQueueSet(
	@Nonnegative int keyGroupId,
	@Nonnegative int keyGroupPrefixBytes,
	@Nonnull RocksDB db,
	@Nonnull ColumnFamilyHandle columnFamilyHandle,
	@Nonnull TypeSerializer<E> byteOrderProducingSerializer,
	@Nonnull DataOutputSerializer outputStream,
	@Nonnull DataInputDeserializer inputStream,
	@Nonnull RocksDBWriteBatchWrapper batchWrapper,
	@Nonnull OrderedByteArraySetCache orderedByteArraySetCache) {
	this.db = db;
	this.columnFamilyHandle = columnFamilyHandle;
	this.byteOrderProducingSerializer = byteOrderProducingSerializer;
	this.batchWrapper = batchWrapper;
	this.outputView = outputStream;
	this.inputView = inputStream;
	this.orderedCache = orderedByteArraySetCache;
	this.allElementsInCache = false;
	this.groupPrefixBytes = createKeyGroupBytes(keyGroupId, keyGroupPrefixBytes);
	this.seekHint = groupPrefixBytes;
	this.internalIndex = HeapPriorityQueueElement.NOT_CONTAINED;
}
Example #4
Source File: MockKeyedStateBackend.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer) {
	return new HeapPriorityQueueSet<>(
		PriorityComparator.forPriorityComparableObjects(),
		KeyExtractorFunction.forKeyedObjects(),
		0,
		keyGroupRange,
		0);
}
Example #5
Source File: RocksDBPriorityQueueSetFactory.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(@Nonnull String stateName, @Nonnull TypeSerializer<T> byteOrderedElementSerializer) {

	final RocksDBKeyedStateBackend.RocksDbKvStateInfo stateCFHandle =
		tryRegisterPriorityQueueMetaInfo(stateName, byteOrderedElementSerializer);

	final ColumnFamilyHandle columnFamilyHandle = stateCFHandle.columnFamilyHandle;

	return new KeyGroupPartitionedPriorityQueue<>(
		KeyExtractorFunction.forKeyedObjects(),
		PriorityComparator.forPriorityComparableObjects(),
		new KeyGroupPartitionedPriorityQueue.PartitionQueueSetFactory<T, RocksDBCachingPriorityQueueSet<T>>() {
			@Nonnull
			@Override
			public RocksDBCachingPriorityQueueSet<T> create(
				int keyGroupId,
				int numKeyGroups,
				@Nonnull KeyExtractorFunction<T> keyExtractor,
				@Nonnull PriorityComparator<T> elementPriorityComparator) {
				TreeOrderedSetCache orderedSetCache = new TreeOrderedSetCache(DEFAULT_CACHES_SIZE);
				return new RocksDBCachingPriorityQueueSet<>(
					keyGroupId,
					keyGroupPrefixBytes,
					db,
					columnFamilyHandle,
					byteOrderedElementSerializer,
					sharedElementOutView,
					sharedElementInView,
					writeBatchWrapper,
					orderedSetCache
				);
			}
		},
		keyGroupRange,
		numberOfKeyGroups);
}
Example #6
Source File: RocksDBKeyedStateBackend.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer) {
	return priorityQueueFactory.create(stateName, byteOrderedElementSerializer);
}
Example #7
Source File: RocksDBPriorityQueueSetFactory.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(@Nonnull String stateName, @Nonnull TypeSerializer<T> byteOrderedElementSerializer) {

	final RocksDBKeyedStateBackend.RocksDbKvStateInfo stateCFHandle =
		tryRegisterPriorityQueueMetaInfo(stateName, byteOrderedElementSerializer);

	final ColumnFamilyHandle columnFamilyHandle = stateCFHandle.columnFamilyHandle;

	return new KeyGroupPartitionedPriorityQueue<>(
		KeyExtractorFunction.forKeyedObjects(),
		PriorityComparator.forPriorityComparableObjects(),
		new KeyGroupPartitionedPriorityQueue.PartitionQueueSetFactory<T, RocksDBCachingPriorityQueueSet<T>>() {
			@Nonnull
			@Override
			public RocksDBCachingPriorityQueueSet<T> create(
				int keyGroupId,
				int numKeyGroups,
				@Nonnull KeyExtractorFunction<T> keyExtractor,
				@Nonnull PriorityComparator<T> elementPriorityComparator) {
				TreeOrderedSetCache orderedSetCache = new TreeOrderedSetCache(DEFAULT_CACHES_SIZE);
				return new RocksDBCachingPriorityQueueSet<>(
					keyGroupId,
					keyGroupPrefixBytes,
					db,
					readOptions,
					columnFamilyHandle,
					byteOrderedElementSerializer,
					sharedElementOutView,
					sharedElementInView,
					writeBatchWrapper,
					orderedSetCache
				);
			}
		},
		keyGroupRange,
		numberOfKeyGroups);
}
Example #8
Source File: ReductionsTest.java    From flink-statefun with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed>
    KeyGroupedInternalPriorityQueue<T> create(
        @Nonnull String stateName, @Nonnull TypeSerializer<T> byteOrderedElementSerializer) {
  throw new UnsupportedOperationException();
}
Example #9
Source File: ReductionsTest.java    From stateful-functions with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed>
    KeyGroupedInternalPriorityQueue<T> create(
        @Nonnull String stateName, @Nonnull TypeSerializer<T> byteOrderedElementSerializer) {
  throw new UnsupportedOperationException();
}
Example #10
Source File: MockKeyedStateBackend.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer) {
	return new HeapPriorityQueueSet<>(
		PriorityComparator.forPriorityComparableObjects(),
		KeyExtractorFunction.forKeyedObjects(),
		0,
		keyGroupRange,
		0);
}
Example #11
Source File: RocksDBKeyedStateBackend.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer) {
	return priorityQueueFactory.create(stateName, byteOrderedElementSerializer);
}
Example #12
Source File: RocksDBPriorityQueueSetFactory.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(@Nonnull String stateName, @Nonnull TypeSerializer<T> byteOrderedElementSerializer) {

	final RocksDBKeyedStateBackend.RocksDbKvStateInfo stateCFHandle =
		tryRegisterPriorityQueueMetaInfo(stateName, byteOrderedElementSerializer);

	final ColumnFamilyHandle columnFamilyHandle = stateCFHandle.columnFamilyHandle;

	return new KeyGroupPartitionedPriorityQueue<>(
		KeyExtractorFunction.forKeyedObjects(),
		PriorityComparator.forPriorityComparableObjects(),
		new KeyGroupPartitionedPriorityQueue.PartitionQueueSetFactory<T, RocksDBCachingPriorityQueueSet<T>>() {
			@Nonnull
			@Override
			public RocksDBCachingPriorityQueueSet<T> create(
				int keyGroupId,
				int numKeyGroups,
				@Nonnull KeyExtractorFunction<T> keyExtractor,
				@Nonnull PriorityComparator<T> elementPriorityComparator) {
				TreeOrderedSetCache orderedSetCache = new TreeOrderedSetCache(DEFAULT_CACHES_SIZE);
				return new RocksDBCachingPriorityQueueSet<>(
					keyGroupId,
					keyGroupPrefixBytes,
					db,
					columnFamilyHandle,
					byteOrderedElementSerializer,
					sharedElementOutView,
					sharedElementInView,
					writeBatchWrapper,
					orderedSetCache
				);
			}
		},
		keyGroupRange,
		numberOfKeyGroups);
}
Example #13
Source File: MockKeyedStateBackend.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer) {
	return new HeapPriorityQueueSet<>(
		PriorityComparator.forPriorityComparableObjects(),
		KeyExtractorFunction.forKeyedObjects(),
		0,
		keyGroupRange,
		0);
}
Example #14
Source File: RocksDBKeyedStateBackend.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Nonnull
@Override
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>
create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer) {
	return priorityQueueFactory.create(stateName, byteOrderedElementSerializer);
}
Example #15
Source File: PriorityQueueSetFactory.java    From flink with Apache License 2.0 2 votes vote down vote up 
/**
 * Creates a {@link KeyGroupedInternalPriorityQueue}.
 *
 * @param stateName                    unique name for associated with this queue.
 * @param byteOrderedElementSerializer a serializer that with a format that is lexicographically ordered in
 *                                     alignment with elementPriorityComparator.
 * @param <T>                          type of the stored elements.
 * @return the queue with the specified unique name.
 */
@Nonnull
<T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T> create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer);
Example #16
Source File: PriorityQueueSetFactory.java    From Flink-CEPplus with Apache License 2.0 2 votes vote down vote up 
/**
 * Creates a {@link KeyGroupedInternalPriorityQueue}.
 *
 * @param stateName                    unique name for associated with this queue.
 * @param byteOrderedElementSerializer a serializer that with a format that is lexicographically ordered in
 *                                     alignment with elementPriorityComparator.
 * @param <T>                          type of the stored elements.
 * @return the queue with the specified unique name.
 */
@Nonnull
<T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T> create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer);
Example #17
Source File: PriorityQueueSetFactory.java    From flink with Apache License 2.0 2 votes vote down vote up 
/**
 * Creates a {@link KeyGroupedInternalPriorityQueue}.
 *
 * @param stateName                    unique name for associated with this queue.
 * @param byteOrderedElementSerializer a serializer that with a format that is lexicographically ordered in
 *                                     alignment with elementPriorityComparator.
 * @param <T>                          type of the stored elements.
 * @return the queue with the specified unique name.
 */
@Nonnull
<T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T> create(
	@Nonnull String stateName,
	@Nonnull TypeSerializer<T> byteOrderedElementSerializer);