Java Code Examples for org.apache.parquet.schema.PrimitiveType#comparator()

The following examples show how to use org.apache.parquet.schema.PrimitiveType#comparator() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.

Example 1

Source File: Statistics.java From parquet-mr with Apache License 2.0

5 votes

Statistics(PrimitiveType type) {
  this.type = type;
  this.comparator = type.comparator();
  this.stringifier = type.stringifier();
  hasNonNullValue = false;
  num_nulls = 0;
}

Example 2

Source File: TestBinaryTruncator.java From parquet-mr with Apache License 2.0

5 votes

private void testTruncator(PrimitiveType type, boolean strict) {
  BinaryTruncator truncator = BinaryTruncator.getTruncator(type);
  Comparator<Binary> comparator = type.comparator();

  checkContract(truncator, comparator, Binary.fromString("aaaaaaaaaa"), strict, strict);
  checkContract(truncator, comparator, Binary.fromString("árvíztűrő tükörfúrógép"), strict, strict);
  checkContract(truncator, comparator, Binary.fromString("aaaaaaaaaa" + UTF8_3BYTES_MAX_CHAR), strict, strict);
  checkContract(truncator, comparator, Binary.fromString("a" + UTF8_3BYTES_MAX_CHAR + UTF8_1BYTE_MAX_CHAR), strict,
      strict);

  checkContract(truncator, comparator,
      Binary.fromConstantByteArray(new byte[] { (byte) 0xFE, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, (byte) 0xFF }), strict,
      strict);

  // Edge case: zero length -> unable to truncate
  checkContract(truncator, comparator, Binary.fromString(""), false, false);
  // Edge case: containing only UTF-8 max characters -> unable to truncate for max
  checkContract(truncator, comparator, Binary.fromString(
      UTF8_1BYTE_MAX_CHAR +
          UTF8_4BYTES_MAX_CHAR +
          UTF8_3BYTES_MAX_CHAR +
          UTF8_4BYTES_MAX_CHAR +
          UTF8_2BYTES_MAX_CHAR +
          UTF8_3BYTES_MAX_CHAR +
          UTF8_3BYTES_MAX_CHAR +
          UTF8_1BYTE_MAX_CHAR +
          UTF8_2BYTES_MAX_CHAR +
          UTF8_3BYTES_MAX_CHAR +
          UTF8_4BYTES_MAX_CHAR),
      strict, false);
  // Edge case: non-UTF-8; max bytes -> unable to truncate for max
  checkContract(
      truncator, comparator,
      binary(0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF),
      strict, false);
}

Example 3

Source File: ColumnIndexBuilder.java From parquet-mr with Apache License 2.0

4 votes

ColumnIndexBase(PrimitiveType type) {
  comparator = type.comparator();
  stringifier = type.stringifier();
}

Example 4

Source File: ColumnIndexValidator.java From parquet-mr with Apache License 2.0

4 votes

Builder(PrimitiveType type) {
  comparator = type.comparator();
  stringifier = type.stringifier();
}