Java Code Examples for org.apache.spark.unsafe.types.UTF8String#fromAddress()
The following examples show how to use
org.apache.spark.unsafe.types.UTF8String#fromAddress() .
You can vote up the ones you like or vote down the ones you don't like,
and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: ArrowSchemaConverter.java From spark-bigquery-connector with Apache License 2.0 | 6 votes |
@Override final UTF8String getUTF8String(int rowId) { accessor.get(rowId, stringResult); if (stringResult.isSet == 0) { return null; } else { ArrowBuf offsets = accessor.getOffsetBuffer(); int index = rowId * VarCharVector.OFFSET_WIDTH; int start = offsets.getInt(index); int end = offsets.getInt(index + VarCharVector.OFFSET_WIDTH); /* Since the result is accessed lazily if the memory address is corrupted we * might lose the data. Might be better to include a byte array. Not doing so * for performance reasons. */ return UTF8String.fromAddress(/* base = */null, stringResult.buffer.memoryAddress() + start, end - start); } }
Example 2
Source File: FlightArrowColumnVector.java From flight-spark-source with Apache License 2.0 | 5 votes |
@Override final UTF8String getUTF8String(int rowId) { accessor.get(rowId, stringResult); if (stringResult.isSet == 0) { return null; } else { return UTF8String.fromAddress(null, stringResult.buffer.memoryAddress() + stringResult.start, stringResult.end - stringResult.start); } }
Example 3
Source File: ArrowVectorAccessors.java From iceberg with Apache License 2.0 | 5 votes |
@Override final UTF8String getUTF8String(int rowId) { vector.get(rowId, stringResult); if (stringResult.isSet == 0) { return null; } else { return UTF8String.fromAddress( null, stringResult.buffer.memoryAddress() + stringResult.start, stringResult.end - stringResult.start); } }
Example 4
Source File: StringsStruct.java From indexr with Apache License 2.0 | 4 votes |
public UTF8String getUTF8String(int index) { int totalIndexLen = (count + 1) << 2; int strOffset = MemoryUtil.getInt(dataAddr + (index << 2)); int nextOffset = MemoryUtil.getInt(dataAddr + ((index + 1) << 2)); return UTF8String.fromAddress(null, dataAddr + totalIndexLen + strOffset, nextOffset - strOffset); }