Java Code Examples for org.apache.parquet.io.api.Binary#fromConstantByteBuffer()

The following examples show how to use org.apache.parquet.io.api.Binary#fromConstantByteBuffer() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: CharValueWriter.java    From presto with Apache License 2.0 5 votes vote down vote up
@Override
public void write(Block block)
{
    for (int i = 0; i < block.getPositionCount(); i++) {
        if (!block.isNull(i)) {
            Slice slice = type.getSlice(block, i);
            Binary binary = Binary.fromConstantByteBuffer(slice.toByteBuffer());
            valuesWriter.writeBytes(binary);
            getStatistics().updateStats(binary);
        }
    }
}
 
Example 2
Source File: ParquetWriterUtil.java    From flink with Apache License 2.0 5 votes vote down vote up
private static Binary timestampToInt96(LocalDateTime time) {
	Timestamp timestamp = Timestamp.valueOf(time);
	long mills = timestamp.getTime();
	int julianDay = (int) ((mills / MILLIS_IN_DAY) + JULIAN_EPOCH_OFFSET_DAYS);
	long nanosOfDay = ((mills % MILLIS_IN_DAY) / 1000) * NANOS_PER_SECOND + timestamp.getNanos();

	ByteBuffer buf = ByteBuffer.allocate(12);
	buf.order(ByteOrder.LITTLE_ENDIAN);
	buf.putLong(nanosOfDay);
	buf.putInt(julianDay);
	buf.flip();
	return Binary.fromConstantByteBuffer(buf);
}
 
Example 3
Source File: NanoTime.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up
public Binary toBinary() {
  ByteBuffer buf = ByteBuffer.allocate(12);
  buf.order(ByteOrder.LITTLE_ENDIAN);
  buf.putLong(timeOfDayNanos);
  buf.putInt(julianDay);
  buf.flip();
  return Binary.fromConstantByteBuffer(buf);
}
 
Example 4
Source File: PlainValuesDictionary.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up
/**
 * Decodes {@link Binary} values from a {@link DictionaryPage}.
 *
 * If the given {@code length} is null, the values will be read as length-
 * prefixed values with a 4-byte little-endian length. If length is not
 * null, it will be used as the length for all fixed-length {@code Binary}
 * values read from the page.
 *
 * @param dictionaryPage a {@code DictionaryPage} of encoded binary values
 * @param length a fixed length of binary arrays, or null if not fixed
 * @throws IOException if there is an exception while decoding the dictionary page
 */
public PlainBinaryDictionary(DictionaryPage dictionaryPage, Integer length) throws IOException {
  super(dictionaryPage);
  final ByteBuffer dictionaryBytes = dictionaryPage.getBytes().toByteBuffer();
  binaryDictionaryContent = new Binary[dictionaryPage.getDictionarySize()];
  // dictionary values are stored in order: size (4 bytes LE) followed by {size} bytes
  int offset = dictionaryBytes.position();
  if (length == null) {
    // dictionary values are stored in order: size (4 bytes LE) followed by {size} bytes
    for (int i = 0; i < binaryDictionaryContent.length; i++) {
      int len = readIntLittleEndian(dictionaryBytes, offset);
      // read the length
      offset += 4;
      // wrap the content in a binary
      binaryDictionaryContent[i] = Binary.fromConstantByteBuffer(dictionaryBytes, offset, len);
      // increment to the next value
      offset += len;
    }
  } else {
    // dictionary values are stored as fixed-length arrays
    Preconditions.checkArgument(length > 0,
        "Invalid byte array length: " + length);
    for (int i = 0; i < binaryDictionaryContent.length; i++) {
      // wrap the content in a Binary
      binaryDictionaryContent[i] = Binary.fromConstantByteBuffer(
          dictionaryBytes, offset, length);
      // increment to the next value
      offset += length;
    }
  }
}
 
Example 5
Source File: DeltaLengthByteArrayValuesReader.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up
@Override
public Binary readBytes() {
  int length = lengthReader.readInteger();
  try {
    return Binary.fromConstantByteBuffer(in.slice(length));
  } catch (IOException e) {
    throw new ParquetDecodingException("Failed to read " + length + " bytes");
  }
}
 
Example 6
Source File: BinaryPlainValuesReader.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up
@Override
public Binary readBytes() {
  try {
    int length = BytesUtils.readIntLittleEndian(in);
    return Binary.fromConstantByteBuffer(in.slice(length));
  } catch (IOException | RuntimeException e) {
    throw new ParquetDecodingException("could not read bytes at offset " + in.position(), e);
  }
}
 
Example 7
Source File: FixedLenByteArrayPlainValuesReader.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up
@Override
public Binary readBytes() {
  try {
    return Binary.fromConstantByteBuffer(in.slice(length));
  } catch (IOException | RuntimeException e) {
    throw new ParquetDecodingException("could not read bytes at offset " + in.position(), e);
  }
}
 
Example 8
Source File: ColumnIndexValidator.java    From parquet-mr with Apache License 2.0 4 votes vote down vote up
@Override
StatValue build(ByteBuffer value) {
  return new Value(Binary.fromConstantByteBuffer(value));
}
 
Example 9
Source File: SliceDictionary.java    From spliceengine with GNU Affero General Public License v3.0 4 votes vote down vote up
@Override
public Binary decodeToBinary(int id) {
    return Binary.fromConstantByteBuffer(slice[id].toByteBuffer());
}
 
Example 10
Source File: SliceDictionary.java    From spliceengine with GNU Affero General Public License v3.0 4 votes vote down vote up
@Override
public Binary decodeToBinary(int id) {
    return Binary.fromConstantByteBuffer(slice[id].toByteBuffer());
}
 
Example 11
Source File: SliceDictionary.java    From spliceengine with GNU Affero General Public License v3.0 4 votes vote down vote up
@Override
public Binary decodeToBinary(int id) {
    return Binary.fromConstantByteBuffer(slice[id].toByteBuffer());
}