Java Code Examples for org.apache.flink.types.StringValue#length()

The following examples show how to use org.apache.flink.types.StringValue#length() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: StringValueUtils.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up
/**
 * Converts the given <code>StringValue</code> into a lower case variant.
 *
 * @param string The string to convert to lower case.
 */
public static void toLowerCase(StringValue string) {
	final char[] chars = string.getCharArray();
	final int len = string.length();

	for (int i = 0; i < len; i++) {
		chars[i] = Character.toLowerCase(chars[i]);
	}
}
 
Example 2
Source File: StringValueUtils.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up
/**
 * Replaces all non-word characters in a string by a given character. The only
 * characters not replaced are the characters that qualify as word characters
 * or digit characters with respect to {@link Character#isLetter(char)} or
 * {@link Character#isDigit(char)}, as well as the underscore character.
 *
 * <p>This operation is intended to simplify strings for counting distinct words.
 *
 * @param string The string value to have the non-word characters replaced.
 * @param replacement The character to use as the replacement.
 */
public static void replaceNonWordChars(StringValue string, char replacement) {
	final char[] chars = string.getCharArray();
	final int len = string.length();

	for (int i = 0; i < len; i++) {
		final char c = chars[i];
		if (!(Character.isLetter(c) || Character.isDigit(c) || c == '_')) {
			chars[i] = replacement;
		}
	}
}
 
Example 3
Source File: StringValueUtils.java    From flink with Apache License 2.0 5 votes vote down vote up
/**
 * Converts the given <code>StringValue</code> into a lower case variant.
 *
 * @param string The string to convert to lower case.
 */
public static void toLowerCase(StringValue string) {
	final char[] chars = string.getCharArray();
	final int len = string.length();

	for (int i = 0; i < len; i++) {
		chars[i] = Character.toLowerCase(chars[i]);
	}
}
 
Example 4
Source File: StringValueUtils.java    From flink with Apache License 2.0 5 votes vote down vote up
/**
 * Replaces all non-word characters in a string by a given character. The only
 * characters not replaced are the characters that qualify as word characters
 * or digit characters with respect to {@link Character#isLetter(char)} or
 * {@link Character#isDigit(char)}, as well as the underscore character.
 *
 * <p>This operation is intended to simplify strings for counting distinct words.
 *
 * @param string The string value to have the non-word characters replaced.
 * @param replacement The character to use as the replacement.
 */
public static void replaceNonWordChars(StringValue string, char replacement) {
	final char[] chars = string.getCharArray();
	final int len = string.length();

	for (int i = 0; i < len; i++) {
		final char c = chars[i];
		if (!(Character.isLetter(c) || Character.isDigit(c) || c == '_')) {
			chars[i] = replacement;
		}
	}
}
 
Example 5
Source File: StringValueUtils.java    From flink with Apache License 2.0 5 votes vote down vote up
/**
 * Converts the given <code>StringValue</code> into a lower case variant.
 *
 * @param string The string to convert to lower case.
 */
public static void toLowerCase(StringValue string) {
	final char[] chars = string.getCharArray();
	final int len = string.length();

	for (int i = 0; i < len; i++) {
		chars[i] = Character.toLowerCase(chars[i]);
	}
}
 
Example 6
Source File: StringValueUtils.java    From flink with Apache License 2.0 5 votes vote down vote up
/**
 * Replaces all non-word characters in a string by a given character. The only
 * characters not replaced are the characters that qualify as word characters
 * or digit characters with respect to {@link Character#isLetter(char)} or
 * {@link Character#isDigit(char)}, as well as the underscore character.
 *
 * <p>This operation is intended to simplify strings for counting distinct words.
 *
 * @param string The string value to have the non-word characters replaced.
 * @param replacement The character to use as the replacement.
 */
public static void replaceNonWordChars(StringValue string, char replacement) {
	final char[] chars = string.getCharArray();
	final int len = string.length();

	for (int i = 0; i < len; i++) {
		final char c = chars[i];
		if (!(Character.isLetter(c) || Character.isDigit(c) || c == '_')) {
			chars[i] = replacement;
		}
	}
}
 
Example 7
Source File: StringValueUtils.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up
/**
 * Sets the string to be tokenized and resets the state of the tokenizer.
 *
 * @param string The string value to be tokenized.
 */
public void setStringToTokenize(StringValue string) {
	this.toTokenize = string;
	this.pos = 0;
	this.limit = string.length();
}
 
Example 8
Source File: StringValueArray.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up
@Override
public boolean add(StringValue value) {
	if (isBounded && position >= boundedLength) {
		return false;
	}

	// up to five bytes storing length
	if (position + 5 > data.length) {
		ensureCapacity(position + 5);
	}

	// update local variable until serialization succeeds
	int newPosition = position;

	// write the length, variable-length encoded
	int len = value.length();

	while (len >= HIGH_BIT) {
		data[newPosition++] = (byte) (len | HIGH_BIT);
		len >>>= 7;
	}
	data[newPosition++] = (byte) len;

	// write the char data, variable-length encoded
	final char[] valueData = value.getCharArray();
	int remainingCapacity = data.length - newPosition;

	len = value.length();
	for (int i = 0; i < len; i++) {
		// up to three bytes storing length
		if (remainingCapacity < 3) {
			ensureCapacity(remainingCapacity + 3);
			remainingCapacity = data.length - newPosition;
		}

		int c = valueData[i];

		while (c >= HIGH_BIT) {
			data[newPosition++] = (byte) (c | HIGH_BIT);
			remainingCapacity--;
			c >>>= 7;
		}
		data[newPosition++] = (byte) c;
		remainingCapacity--;
	}

	length++;
	position = newPosition;

	return true;
}
 
Example 9
Source File: StringValueUtils.java    From flink with Apache License 2.0 4 votes vote down vote up
/**
 * Sets the string to be tokenized and resets the state of the tokenizer.
 *
 * @param string The string value to be tokenized.
 */
public void setStringToTokenize(StringValue string) {
	this.toTokenize = string;
	this.pos = 0;
	this.limit = string.length();
}
 
Example 10
Source File: StringValueArray.java    From flink with Apache License 2.0 4 votes vote down vote up
@Override
public boolean add(StringValue value) {
	if (isBounded && position >= boundedLength) {
		return false;
	}

	// up to five bytes storing length
	if (position + 5 > data.length) {
		ensureCapacity(position + 5);
	}

	// update local variable until serialization succeeds
	int newPosition = position;

	// write the length, variable-length encoded
	int len = value.length();

	while (len >= HIGH_BIT) {
		data[newPosition++] = (byte) (len | HIGH_BIT);
		len >>>= 7;
	}
	data[newPosition++] = (byte) len;

	// write the char data, variable-length encoded
	final char[] valueData = value.getCharArray();
	int remainingCapacity = data.length - newPosition;

	len = value.length();
	for (int i = 0; i < len; i++) {
		// up to three bytes storing length
		if (remainingCapacity < 3) {
			ensureCapacity(remainingCapacity + 3);
			remainingCapacity = data.length - newPosition;
		}

		int c = valueData[i];

		while (c >= HIGH_BIT) {
			data[newPosition++] = (byte) (c | HIGH_BIT);
			remainingCapacity--;
			c >>>= 7;
		}
		data[newPosition++] = (byte) c;
		remainingCapacity--;
	}

	length++;
	position = newPosition;

	return true;
}
 
Example 11
Source File: StringValueUtils.java    From flink with Apache License 2.0 4 votes vote down vote up
/**
 * Sets the string to be tokenized and resets the state of the tokenizer.
 *
 * @param string The string value to be tokenized.
 */
public void setStringToTokenize(StringValue string) {
	this.toTokenize = string;
	this.pos = 0;
	this.limit = string.length();
}
 
Example 12
Source File: StringValueArray.java    From flink with Apache License 2.0 4 votes vote down vote up
@Override
public boolean add(StringValue value) {
	if (isBounded && position >= boundedLength) {
		return false;
	}

	// up to five bytes storing length
	if (position + 5 > data.length) {
		ensureCapacity(position + 5);
	}

	// update local variable until serialization succeeds
	int newPosition = position;

	// write the length, variable-length encoded
	int len = value.length();

	while (len >= HIGH_BIT) {
		data[newPosition++] = (byte) (len | HIGH_BIT);
		len >>>= 7;
	}
	data[newPosition++] = (byte) len;

	// write the char data, variable-length encoded
	final char[] valueData = value.getCharArray();
	int remainingCapacity = data.length - newPosition;

	len = value.length();
	for (int i = 0; i < len; i++) {
		// up to three bytes storing length
		if (remainingCapacity < 3) {
			ensureCapacity(remainingCapacity + 3);
			remainingCapacity = data.length - newPosition;
		}

		int c = valueData[i];

		while (c >= HIGH_BIT) {
			data[newPosition++] = (byte) (c | HIGH_BIT);
			remainingCapacity--;
			c >>>= 7;
		}
		data[newPosition++] = (byte) c;
		remainingCapacity--;
	}

	length++;
	position = newPosition;

	return true;
}