Java Code Examples for org.apache.spark.unsafe.types.UTF8String#fromAddress()

The following examples show how to use org.apache.spark.unsafe.types.UTF8String#fromAddress() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: ArrowSchemaConverter.java    From spark-bigquery-connector with Apache License 2.0 6 votes vote down vote up 
@Override
final UTF8String getUTF8String(int rowId) {
  accessor.get(rowId, stringResult);
  if (stringResult.isSet == 0) {
    return null;
  } else {
    ArrowBuf offsets = accessor.getOffsetBuffer();
    int index = rowId * VarCharVector.OFFSET_WIDTH;
    int start = offsets.getInt(index);
    int end = offsets.getInt(index + VarCharVector.OFFSET_WIDTH);

    /* Since the result is accessed lazily if the memory address is corrupted we
     * might lose the data. Might be better to include a byte array. Not doing so
     * for performance reasons.
     */
    return UTF8String.fromAddress(/* base = */null,
        stringResult.buffer.memoryAddress() + start,
        end - start);
  }
}
Example 2
Source File: FlightArrowColumnVector.java    From flight-spark-source with Apache License 2.0 5 votes vote down vote up 
@Override
final UTF8String getUTF8String(int rowId) {
  accessor.get(rowId, stringResult);
  if (stringResult.isSet == 0) {
    return null;
  } else {
    return UTF8String.fromAddress(null,
      stringResult.buffer.memoryAddress() + stringResult.start,
      stringResult.end - stringResult.start);
  }
}
Example 3
Source File: ArrowVectorAccessors.java    From iceberg with Apache License 2.0 5 votes vote down vote up 
@Override
final UTF8String getUTF8String(int rowId) {
  vector.get(rowId, stringResult);
  if (stringResult.isSet == 0) {
    return null;
  } else {
    return UTF8String.fromAddress(
        null,
        stringResult.buffer.memoryAddress() + stringResult.start,
        stringResult.end - stringResult.start);
  }
}
Example 4
Source File: StringsStruct.java    From indexr with Apache License 2.0 4 votes vote down vote up 
public UTF8String getUTF8String(int index) {
    int totalIndexLen = (count + 1) << 2;
    int strOffset = MemoryUtil.getInt(dataAddr + (index << 2));
    int nextOffset = MemoryUtil.getInt(dataAddr + ((index + 1) << 2));
    return UTF8String.fromAddress(null, dataAddr + totalIndexLen + strOffset, nextOffset - strOffset);
}