Java Code Examples for org.apache.parquet.io.api.Binary#fromConstantByteBuffer()
The following examples show how to use
org.apache.parquet.io.api.Binary#fromConstantByteBuffer() .
You can vote up the ones you like or vote down the ones you don't like,
and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: CharValueWriter.java From presto with Apache License 2.0 | 5 votes |
@Override public void write(Block block) { for (int i = 0; i < block.getPositionCount(); i++) { if (!block.isNull(i)) { Slice slice = type.getSlice(block, i); Binary binary = Binary.fromConstantByteBuffer(slice.toByteBuffer()); valuesWriter.writeBytes(binary); getStatistics().updateStats(binary); } } }
Example 2
Source File: ParquetWriterUtil.java From flink with Apache License 2.0 | 5 votes |
private static Binary timestampToInt96(LocalDateTime time) { Timestamp timestamp = Timestamp.valueOf(time); long mills = timestamp.getTime(); int julianDay = (int) ((mills / MILLIS_IN_DAY) + JULIAN_EPOCH_OFFSET_DAYS); long nanosOfDay = ((mills % MILLIS_IN_DAY) / 1000) * NANOS_PER_SECOND + timestamp.getNanos(); ByteBuffer buf = ByteBuffer.allocate(12); buf.order(ByteOrder.LITTLE_ENDIAN); buf.putLong(nanosOfDay); buf.putInt(julianDay); buf.flip(); return Binary.fromConstantByteBuffer(buf); }
Example 3
Source File: NanoTime.java From parquet-mr with Apache License 2.0 | 5 votes |
public Binary toBinary() { ByteBuffer buf = ByteBuffer.allocate(12); buf.order(ByteOrder.LITTLE_ENDIAN); buf.putLong(timeOfDayNanos); buf.putInt(julianDay); buf.flip(); return Binary.fromConstantByteBuffer(buf); }
Example 4
Source File: PlainValuesDictionary.java From parquet-mr with Apache License 2.0 | 5 votes |
/** * Decodes {@link Binary} values from a {@link DictionaryPage}. * * If the given {@code length} is null, the values will be read as length- * prefixed values with a 4-byte little-endian length. If length is not * null, it will be used as the length for all fixed-length {@code Binary} * values read from the page. * * @param dictionaryPage a {@code DictionaryPage} of encoded binary values * @param length a fixed length of binary arrays, or null if not fixed * @throws IOException if there is an exception while decoding the dictionary page */ public PlainBinaryDictionary(DictionaryPage dictionaryPage, Integer length) throws IOException { super(dictionaryPage); final ByteBuffer dictionaryBytes = dictionaryPage.getBytes().toByteBuffer(); binaryDictionaryContent = new Binary[dictionaryPage.getDictionarySize()]; // dictionary values are stored in order: size (4 bytes LE) followed by {size} bytes int offset = dictionaryBytes.position(); if (length == null) { // dictionary values are stored in order: size (4 bytes LE) followed by {size} bytes for (int i = 0; i < binaryDictionaryContent.length; i++) { int len = readIntLittleEndian(dictionaryBytes, offset); // read the length offset += 4; // wrap the content in a binary binaryDictionaryContent[i] = Binary.fromConstantByteBuffer(dictionaryBytes, offset, len); // increment to the next value offset += len; } } else { // dictionary values are stored as fixed-length arrays Preconditions.checkArgument(length > 0, "Invalid byte array length: " + length); for (int i = 0; i < binaryDictionaryContent.length; i++) { // wrap the content in a Binary binaryDictionaryContent[i] = Binary.fromConstantByteBuffer( dictionaryBytes, offset, length); // increment to the next value offset += length; } } }
Example 5
Source File: DeltaLengthByteArrayValuesReader.java From parquet-mr with Apache License 2.0 | 5 votes |
@Override public Binary readBytes() { int length = lengthReader.readInteger(); try { return Binary.fromConstantByteBuffer(in.slice(length)); } catch (IOException e) { throw new ParquetDecodingException("Failed to read " + length + " bytes"); } }
Example 6
Source File: BinaryPlainValuesReader.java From parquet-mr with Apache License 2.0 | 5 votes |
@Override public Binary readBytes() { try { int length = BytesUtils.readIntLittleEndian(in); return Binary.fromConstantByteBuffer(in.slice(length)); } catch (IOException | RuntimeException e) { throw new ParquetDecodingException("could not read bytes at offset " + in.position(), e); } }
Example 7
Source File: FixedLenByteArrayPlainValuesReader.java From parquet-mr with Apache License 2.0 | 5 votes |
@Override public Binary readBytes() { try { return Binary.fromConstantByteBuffer(in.slice(length)); } catch (IOException | RuntimeException e) { throw new ParquetDecodingException("could not read bytes at offset " + in.position(), e); } }
Example 8
Source File: ColumnIndexValidator.java From parquet-mr with Apache License 2.0 | 4 votes |
@Override StatValue build(ByteBuffer value) { return new Value(Binary.fromConstantByteBuffer(value)); }
Example 9
Source File: SliceDictionary.java From spliceengine with GNU Affero General Public License v3.0 | 4 votes |
@Override public Binary decodeToBinary(int id) { return Binary.fromConstantByteBuffer(slice[id].toByteBuffer()); }
Example 10
Source File: SliceDictionary.java From spliceengine with GNU Affero General Public License v3.0 | 4 votes |
@Override public Binary decodeToBinary(int id) { return Binary.fromConstantByteBuffer(slice[id].toByteBuffer()); }
Example 11
Source File: SliceDictionary.java From spliceengine with GNU Affero General Public License v3.0 | 4 votes |
@Override public Binary decodeToBinary(int id) { return Binary.fromConstantByteBuffer(slice[id].toByteBuffer()); }