Java Code Examples for org.apache.flink.streaming.api.windowing.time.Time#toMilliseconds()

The following examples show how to use org.apache.flink.streaming.api.windowing.time.Time#toMilliseconds() . These examples are extracted from open source projects. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source Project: Flink-CEPplus   File: KeyedStream.java    License: Apache License 2.0 6 votes vote down vote up
/**
 * Specifies the time boundaries over which the join operation works, so that
 * <pre>leftElement.timestamp + lowerBound <= rightElement.timestamp <= leftElement.timestamp + upperBound</pre>
 * By default both the lower and the upper bound are inclusive. This can be configured
 * with {@link IntervalJoined#lowerBoundExclusive()} and
 * {@link IntervalJoined#upperBoundExclusive()}
 *
 * @param lowerBound The lower bound. Needs to be smaller than or equal to the upperBound
 * @param upperBound The upper bound. Needs to be bigger than or equal to the lowerBound
 */
@PublicEvolving
public IntervalJoined<T1, T2, KEY> between(Time lowerBound, Time upperBound) {

	TimeCharacteristic timeCharacteristic =
		streamOne.getExecutionEnvironment().getStreamTimeCharacteristic();

	if (timeCharacteristic != TimeCharacteristic.EventTime) {
		throw new UnsupportedTimeCharacteristicException("Time-bounded stream joins are only supported in event time");
	}

	checkNotNull(lowerBound, "A lower bound needs to be provided for a time-bounded join");
	checkNotNull(upperBound, "An upper bound needs to be provided for a time-bounded join");

	return new IntervalJoined<>(
		streamOne,
		streamTwo,
		lowerBound.toMilliseconds(),
		upperBound.toMilliseconds(),
		true,
		true
	);
}
 
Example 2
Source Project: flink   File: KeyedStream.java    License: Apache License 2.0 6 votes vote down vote up
/**
 * Specifies the time boundaries over which the join operation works, so that
 * <pre>leftElement.timestamp + lowerBound <= rightElement.timestamp <= leftElement.timestamp + upperBound</pre>
 * By default both the lower and the upper bound are inclusive. This can be configured
 * with {@link IntervalJoined#lowerBoundExclusive()} and
 * {@link IntervalJoined#upperBoundExclusive()}
 *
 * @param lowerBound The lower bound. Needs to be smaller than or equal to the upperBound
 * @param upperBound The upper bound. Needs to be bigger than or equal to the lowerBound
 */
@PublicEvolving
public IntervalJoined<T1, T2, KEY> between(Time lowerBound, Time upperBound) {

	TimeCharacteristic timeCharacteristic =
		streamOne.getExecutionEnvironment().getStreamTimeCharacteristic();

	if (timeCharacteristic != TimeCharacteristic.EventTime) {
		throw new UnsupportedTimeCharacteristicException("Time-bounded stream joins are only supported in event time");
	}

	checkNotNull(lowerBound, "A lower bound needs to be provided for a time-bounded join");
	checkNotNull(upperBound, "An upper bound needs to be provided for a time-bounded join");

	return new IntervalJoined<>(
		streamOne,
		streamTwo,
		lowerBound.toMilliseconds(),
		upperBound.toMilliseconds(),
		true,
		true
	);
}
 
Example 3
Source Project: Flink-CEPplus   File: NFACompiler.java    License: Apache License 2.0 5 votes vote down vote up
/**
 * Creates all the states between Start and Final state.
 *
 * @param sinkState the state that last state should point to (always the Final state)
 * @return the next state after Start in the resulting graph
 */
private State<T> createMiddleStates(final State<T> sinkState) {
	State<T> lastSink = sinkState;
	while (currentPattern.getPrevious() != null) {

		if (currentPattern.getQuantifier().getConsumingStrategy() == Quantifier.ConsumingStrategy.NOT_FOLLOW) {
			//skip notFollow patterns, they are converted into edge conditions
		} else if (currentPattern.getQuantifier().getConsumingStrategy() == Quantifier.ConsumingStrategy.NOT_NEXT) {
			final State<T> notNext = createState(currentPattern.getName(), State.StateType.Normal);
			final IterativeCondition<T> notCondition = getTakeCondition(currentPattern);
			final State<T> stopState = createStopState(notCondition, currentPattern.getName());

			if (lastSink.isFinal()) {
				//so that the proceed to final is not fired
				notNext.addIgnore(lastSink, new RichNotCondition<>(notCondition));
			} else {
				notNext.addProceed(lastSink, new RichNotCondition<>(notCondition));
			}
			notNext.addProceed(stopState, notCondition);
			lastSink = notNext;
		} else {
			lastSink = convertPattern(lastSink);
		}

		// we traverse the pattern graph backwards
		followingPattern = currentPattern;
		currentPattern = currentPattern.getPrevious();

		final Time currentWindowTime = currentPattern.getWindowTime();
		if (currentWindowTime != null && currentWindowTime.toMilliseconds() < windowTime) {
			// the window time is the global minimum of all window times of each state
			windowTime = currentWindowTime.toMilliseconds();
		}
	}
	return lastSink;
}
 
Example 4
Source Project: Flink-CEPplus   File: WindowedStream.java    License: Apache License 2.0 5 votes vote down vote up
/**
 * Sets the time by which elements are allowed to be late. Elements that
 * arrive behind the watermark by more than the specified time will be dropped.
 * By default, the allowed lateness is {@code 0L}.
 *
 * <p>Setting an allowed lateness is only valid for event-time windows.
 */
@PublicEvolving
public WindowedStream<T, K, W> allowedLateness(Time lateness) {
	final long millis = lateness.toMilliseconds();
	checkArgument(millis >= 0, "The allowed lateness cannot be negative.");

	this.allowedLateness = millis;
	return this;
}
 
Example 5
Source Project: flink   File: WindowedStream.java    License: Apache License 2.0 5 votes vote down vote up
/**
 * Sets the time by which elements are allowed to be late. Elements that
 * arrive behind the watermark by more than the specified time will be dropped.
 * By default, the allowed lateness is {@code 0L}.
 *
 * <p>Setting an allowed lateness is only valid for event-time windows.
 */
@PublicEvolving
public WindowedStream<T, K, W> allowedLateness(Time lateness) {
	final long millis = lateness.toMilliseconds();
	checkArgument(millis >= 0, "The allowed lateness cannot be negative.");

	this.allowedLateness = millis;
	return this;
}
 
Example 6
Source Project: flink   File: AllWindowedStream.java    License: Apache License 2.0 5 votes vote down vote up
/**
 * Sets the time by which elements are allowed to be late. Elements that
 * arrive behind the watermark by more than the specified time will be dropped.
 * By default, the allowed lateness is {@code 0L}.
 *
 * <p>Setting an allowed lateness is only valid for event-time windows.
 */
@PublicEvolving
public AllWindowedStream<T, W> allowedLateness(Time lateness) {
	final long millis = lateness.toMilliseconds();
	checkArgument(millis >= 0, "The allowed lateness cannot be negative.");

	this.allowedLateness = millis;
	return this;
}
 
Example 7
Source Project: flink   File: AllWindowedStream.java    License: Apache License 2.0 5 votes vote down vote up
/**
 * Sets the time by which elements are allowed to be late. Elements that
 * arrive behind the watermark by more than the specified time will be dropped.
 * By default, the allowed lateness is {@code 0L}.
 *
 * <p>Setting an allowed lateness is only valid for event-time windows.
 */
@PublicEvolving
public AllWindowedStream<T, W> allowedLateness(Time lateness) {
	final long millis = lateness.toMilliseconds();
	checkArgument(millis >= 0, "The allowed lateness cannot be negative.");

	this.allowedLateness = millis;
	return this;
}
 
Example 8
Source Project: flink   File: NFACompiler.java    License: Apache License 2.0 5 votes vote down vote up
/**
 * Creates all the states between Start and Final state.
 *
 * @param sinkState the state that last state should point to (always the Final state)
 * @return the next state after Start in the resulting graph
 */
private State<T> createMiddleStates(final State<T> sinkState) {
	State<T> lastSink = sinkState;
	while (currentPattern.getPrevious() != null) {

		if (currentPattern.getQuantifier().getConsumingStrategy() == Quantifier.ConsumingStrategy.NOT_FOLLOW) {
			//skip notFollow patterns, they are converted into edge conditions
		} else if (currentPattern.getQuantifier().getConsumingStrategy() == Quantifier.ConsumingStrategy.NOT_NEXT) {
			final State<T> notNext = createState(currentPattern.getName(), State.StateType.Normal);
			final IterativeCondition<T> notCondition = getTakeCondition(currentPattern);
			final State<T> stopState = createStopState(notCondition, currentPattern.getName());

			if (lastSink.isFinal()) {
				//so that the proceed to final is not fired
				notNext.addIgnore(lastSink, new RichNotCondition<>(notCondition));
			} else {
				notNext.addProceed(lastSink, new RichNotCondition<>(notCondition));
			}
			notNext.addProceed(stopState, notCondition);
			lastSink = notNext;
		} else {
			lastSink = convertPattern(lastSink);
		}

		// we traverse the pattern graph backwards
		followingPattern = currentPattern;
		currentPattern = currentPattern.getPrevious();

		final Time currentWindowTime = currentPattern.getWindowTime();
		if (currentWindowTime != null && currentWindowTime.toMilliseconds() < windowTime.orElse(Long.MAX_VALUE)) {
			// the window time is the global minimum of all window times of each state
			windowTime = Optional.of(currentWindowTime.toMilliseconds());
		}
	}
	return lastSink;
}
 
Example 9
Source Project: flink   File: TumblingEventTimeWindows.java    License: Apache License 2.0 2 votes vote down vote up
/**
 * Creates a new {@code TumblingEventTimeWindows} {@link WindowAssigner} that assigns
 * elements to time windows based on the element timestamp and offset.
 *
 * <p>For example, if you want window a stream by hour,but window begins at the 15th minutes
 * of each hour, you can use {@code of(Time.hours(1),Time.minutes(15))},then you will get
 * time windows start at 0:15:00,1:15:00,2:15:00,etc.
 *
 * <p>Rather than that,if you are living in somewhere which is not using UTC±00:00 time,
 * such as China which is using UTC+08:00,and you want a time window with size of one day,
 * and window begins at every 00:00:00 of local time,you may use {@code of(Time.days(1),Time.hours(-8))}.
 * The parameter of offset is {@code Time.hours(-8))} since UTC+08:00 is 8 hours earlier than UTC time.
 *
 * @param size The size of the generated windows.
 * @param offset The offset which window start would be shifted by.
 * @return The time policy.
 */
public static TumblingEventTimeWindows of(Time size, Time offset) {
	return new TumblingEventTimeWindows(size.toMilliseconds(), offset.toMilliseconds());
}
 
Example 10
/**
 * Creates a new {@code TumblingProcessingTimeWindows} {@link WindowAssigner} that assigns
 * elements to time windows based on the element timestamp and offset.
 *
 * <p>For example, if you want window a stream by hour,but window begins at the 15th minutes
 * of each hour, you can use {@code of(Time.hours(1),Time.minutes(15))},then you will get
 * time windows start at 0:15:00,1:15:00,2:15:00,etc.
 *
 * <p>Rather than that,if you are living in somewhere which is not using UTC±00:00 time,
 * such as China which is using UTC+08:00,and you want a time window with size of one day,
 * and window begins at every 00:00:00 of local time,you may use {@code of(Time.days(1),Time.hours(-8))}.
 * The parameter of offset is {@code Time.hours(-8))} since UTC+08:00 is 8 hours earlier than UTC time.
 *
 * @param size The size of the generated windows.
 * @param offset The offset which window start would be shifted by.
 * @return The time policy.
 */
public static TumblingProcessingTimeWindows of(Time size, Time offset) {
	return new TumblingProcessingTimeWindows(size.toMilliseconds(), offset.toMilliseconds(), WindowStagger.ALIGNED);
}
 
Example 11
Source Project: flink   File: SlidingEventTimeWindows.java    License: Apache License 2.0 2 votes vote down vote up
/**
 * Creates a new {@code SlidingEventTimeWindows} {@link WindowAssigner} that assigns
 * elements to sliding time windows based on the element timestamp.
 *
 * @param size The size of the generated windows.
 * @param slide The slide interval of the generated windows.
 * @return The time policy.
 */
public static SlidingEventTimeWindows of(Time size, Time slide) {
	return new SlidingEventTimeWindows(size.toMilliseconds(), slide.toMilliseconds(), 0);
}
 
Example 12
Source Project: flink   File: SlidingTimeWindows.java    License: Apache License 2.0 2 votes vote down vote up
/**
 * Creates a new {@code SlidingTimeWindows} {@link WindowAssigner} that assigns
 * elements to sliding time windows based on the element timestamp.
 *
 * @deprecated Please use {@link SlidingEventTimeWindows#of(Time, Time)}.
 *
 * @param size The size of the generated windows.
 * @param slide The slide interval of the generated windows.
 * @return The time policy.
 */
@Deprecated()
public static SlidingTimeWindows of(Time size, Time slide) {
	return new SlidingTimeWindows(size.toMilliseconds(), slide.toMilliseconds());
}
 
Example 13
/**
 * Creates a new {@code SessionWindows} {@link WindowAssigner} that assigns
 * elements to sessions based on the element timestamp.
 *
 * @param size The session timeout, i.e. the time gap between sessions
 * @return The policy.
 */
public static ProcessingTimeSessionWindows withGap(Time size) {
	return new ProcessingTimeSessionWindows(size.toMilliseconds());
}
 
Example 14
Source Project: flink   File: SlidingEventTimeWindows.java    License: Apache License 2.0 2 votes vote down vote up
/**
 * Creates a new {@code SlidingEventTimeWindows} {@link WindowAssigner} that assigns
 * elements to time windows based on the element timestamp and offset.
 *
 * <p>For example, if you want window a stream by hour,but window begins at the 15th minutes
 * of each hour, you can use {@code of(Time.hours(1),Time.minutes(15))},then you will get
 * time windows start at 0:15:00,1:15:00,2:15:00,etc.
 *
 * <p>Rather than that,if you are living in somewhere which is not using UTC±00:00 time,
 * such as China which is using UTC+08:00,and you want a time window with size of one day,
 * and window begins at every 00:00:00 of local time,you may use {@code of(Time.days(1),Time.hours(-8))}.
 * The parameter of offset is {@code Time.hours(-8))} since UTC+08:00 is 8 hours earlier than UTC time.
 *
 * @param size The size of the generated windows.
 * @param slide  The slide interval of the generated windows.
 * @param offset The offset which window start would be shifted by.
 * @return The time policy.
 */
public static SlidingEventTimeWindows of(Time size, Time slide, Time offset) {
	return new SlidingEventTimeWindows(size.toMilliseconds(), slide.toMilliseconds(), offset.toMilliseconds());
}
 
Example 15
/**
 * Creates a new {@code SlidingEventTimeWindows} {@link WindowAssigner} that assigns
 * elements to time windows based on the element timestamp and offset.
 *
 * <p>For example, if you want window a stream by hour,but window begins at the 15th minutes
 * of each hour, you can use {@code of(Time.hours(1),Time.minutes(15))},then you will get
 * time windows start at 0:15:00,1:15:00,2:15:00,etc.
 *
 * <p>Rather than that,if you are living in somewhere which is not using UTC±00:00 time,
 * such as China which is using UTC+08:00,and you want a time window with size of one day,
 * and window begins at every 00:00:00 of local time,you may use {@code of(Time.days(1),Time.hours(-8))}.
 * The parameter of offset is {@code Time.hours(-8))} since UTC+08:00 is 8 hours earlier than UTC time.
 *
 * @param size The size of the generated windows.
 * @param slide  The slide interval of the generated windows.
 * @param offset The offset which window start would be shifted by.
 * @return The time policy.
 */
public static SlidingEventTimeWindows of(Time size, Time slide, Time offset) {
	return new SlidingEventTimeWindows(size.toMilliseconds(), slide.toMilliseconds(), offset.toMilliseconds());
}
 
Example 16
/**
 * Creates a new {@code SlidingProcessingTimeWindows} {@link WindowAssigner} that assigns
 * elements to time windows based on the element timestamp and offset.
 *
 * <p>For example, if you want window a stream by hour,but window begins at the 15th minutes
 * of each hour, you can use {@code of(Time.hours(1),Time.minutes(15))},then you will get
 * time windows start at 0:15:00,1:15:00,2:15:00,etc.
 *
 * <p>Rather than that,if you are living in somewhere which is not using UTC±00:00 time,
 * such as China which is using UTC+08:00,and you want a time window with size of one day,
 * and window begins at every 00:00:00 of local time,you may use {@code of(Time.days(1),Time.hours(-8))}.
 * The parameter of offset is {@code Time.hours(-8))} since UTC+08:00 is 8 hours earlier than UTC time.
 *
 * @param size The size of the generated windows.
 * @param slide  The slide interval of the generated windows.
 * @param offset The offset which window start would be shifted by.
 * @return The time policy.
 */
public static SlidingProcessingTimeWindows of(Time size, Time slide, Time offset) {
	return new SlidingProcessingTimeWindows(size.toMilliseconds(), slide.toMilliseconds(), offset.toMilliseconds());
}
 
Example 17
/**
 * Creates a new {@code TumblingProcessingTimeWindows} {@link WindowAssigner} that assigns
 * elements to time windows based on the element timestamp and offset.
 *
 * <p>For example, if you want window a stream by hour,but window begins at the 15th minutes
 * of each hour, you can use {@code of(Time.hours(1),Time.minutes(15))},then you will get
 * time windows start at 0:15:00,1:15:00,2:15:00,etc.
 *
 * <p>Rather than that,if you are living in somewhere which is not using UTC±00:00 time,
 * such as China which is using UTC+08:00,and you want a time window with size of one day,
 * and window begins at every 00:00:00 of local time,you may use {@code of(Time.days(1),Time.hours(-8))}.
 * The parameter of offset is {@code Time.hours(-8))} since UTC+08:00 is 8 hours earlier than UTC time.
 *
 * @param size The size of the generated windows.
 * @param offset The offset which window start would be shifted by.
 * @return The time policy.
 */
public static TumblingProcessingTimeWindows of(Time size, Time offset) {
	return new TumblingProcessingTimeWindows(size.toMilliseconds(), offset.toMilliseconds());
}
 
Example 18
Source Project: Flink-CEPplus   File: TimeEvictor.java    License: Apache License 2.0 2 votes vote down vote up
/**
 * Creates a {@code TimeEvictor} that keeps the given number of elements.
 * Eviction is done before the window function.
 *
 * @param windowSize The amount of time for which to keep elements.
 */
public static <W extends Window> TimeEvictor<W> of(Time windowSize) {
	return new TimeEvictor<>(windowSize.toMilliseconds());
}
 
Example 19
Source Project: flink   File: TumblingTimeWindows.java    License: Apache License 2.0 2 votes vote down vote up
/**
 * Creates a new {@code TumblingTimeWindows} {@link WindowAssigner} that assigns
 * elements to time windows based on the element timestamp.
 *
 * @deprecated Please use {@link TumblingEventTimeWindows#of(Time)}.
 *
 * @param size The size of the generated windows.
 * @return The time policy.
 */
@Deprecated()
public static TumblingTimeWindows of(Time size) {
	return new TumblingTimeWindows(size.toMilliseconds());
}
 
Example 20
Source Project: flink   File: TimeEvictor.java    License: Apache License 2.0 2 votes vote down vote up
/**
 * Creates a {@code TimeEvictor} that keeps the given number of elements.
 * Eviction is done before/after the window function based on the value of doEvictAfter.
 *
 * @param windowSize The amount of time for which to keep elements.
 * @param doEvictAfter Whether eviction is done after window function.
    */
public static <W extends Window> TimeEvictor<W> of(Time windowSize, boolean doEvictAfter) {
	return new TimeEvictor<>(windowSize.toMilliseconds(), doEvictAfter);
}