Java Code Examples for org.apache.lucene.util.RamUsageEstimator#NUM_BYTES_OBJECT_REF

The following examples show how to use org.apache.lucene.util.RamUsageEstimator#NUM_BYTES_OBJECT_REF . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: CompositeBytesReference.java    From crate with Apache License 2.0 6 votes vote down vote up
public CompositeBytesReference(BytesReference... references) {
    this.references = Objects.requireNonNull(references, "references must not be null");
    this.offsets = new int[references.length];
    long ramBytesUsed = 0;
    int offset = 0;
    for (int i = 0; i < references.length; i++) {
        BytesReference reference = references[i];
        if (reference == null) {
            throw new IllegalArgumentException("references must not be null");
        }
        offsets[i] = offset; // we use the offsets to seek into the right BytesReference for random access and slicing
        offset += reference.length();
        ramBytesUsed += reference.ramBytesUsed();
    }
    this.ramBytesUsed = ramBytesUsed
        + (Integer.BYTES * offsets.length + RamUsageEstimator.NUM_BYTES_ARRAY_HEADER) // offsets
        + (references.length * RamUsageEstimator.NUM_BYTES_OBJECT_REF + RamUsageEstimator.NUM_BYTES_ARRAY_HEADER) // references
        + Integer.BYTES // length
        + Long.BYTES; // ramBytesUsed
    length = offset;
}
 
Example 2
Source File: ByteBuffersDataInput.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
@Override
public long ramBytesUsed() {
  // Return a rough estimation for allocated blocks. Note that we do not make
  // any special distinction for what the type of buffer is (direct vs. heap-based).
  return RamUsageEstimator.NUM_BYTES_OBJECT_REF * blocks.length + 
         Arrays.stream(blocks).mapToLong(buf -> buf.capacity()).sum();
}
 
Example 3
Source File: ByteBuffersDataOutput.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
@Override
public long ramBytesUsed() {
  // Return a rough estimation for allocated blocks. Note that we do not make
  // any special distinction for direct memory buffers.
  return RamUsageEstimator.NUM_BYTES_OBJECT_REF * blocks.size() + 
         blocks.stream().mapToLong(buf -> buf.capacity()).sum();
}
 
Example 4
Source File: Automaton.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
@Override
public long ramBytesUsed() {
  // TODO: BitSet RAM usage (isAccept.size()/8) isn't fully accurate...
  return RamUsageEstimator.NUM_BYTES_OBJECT_HEADER + RamUsageEstimator.sizeOf(states) + RamUsageEstimator.sizeOf(transitions) +
    RamUsageEstimator.NUM_BYTES_OBJECT_HEADER + (isAccept.size() / 8) + RamUsageEstimator.NUM_BYTES_OBJECT_REF +
    2 * RamUsageEstimator.NUM_BYTES_OBJECT_REF +
    3 * Integer.BYTES +
    1;
}
 
Example 5
Source File: SimpleTextFieldsReader.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
@Override
public synchronized long ramBytesUsed() {
  long sizeInBytes = BASE_RAM_BYTES_USED + fields.size() * 2 * RamUsageEstimator.NUM_BYTES_OBJECT_REF;
  for(SimpleTextTerms simpleTextTerms : termsCache.values()) {
    sizeInBytes += (simpleTextTerms!=null) ? simpleTextTerms.ramBytesUsed() : 0;
  }
  return sizeInBytes;
}
 
Example 6
Source File: FieldCacheImpl.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
@Override
public long ramBytesUsed() {
  long base = RamUsageEstimator.NUM_BYTES_OBJECT_REF;
  if (bits instanceof Bits.MatchAllBits || bits instanceof Bits.MatchNoBits) {
    return base;
  } else {
    return base + (bits.length() >>> 3);
  }
}
 
Example 7
Source File: BaseIndexFileFormatTestCase.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
public long accumulateObject(Object o, long shallowSize, Map<java.lang.reflect.Field, Object> fieldValues, Collection<Object> queue) {
  for (Class<?> clazz = o.getClass(); clazz != null; clazz = clazz.getSuperclass()) {
    if (EXCLUDED_CLASSES.contains(clazz) && o != root) {
      return 0;
    }
  }
  // we have no way to estimate the size of these things in codecs although
  // something like a Collections.newSetFromMap(new HashMap<>()) uses quite
  // some memory... So for now the test ignores the overhead of such
  // collections but can we do better?
  long v;
  if (o instanceof Collection) {
    Collection<?> coll = (Collection<?>) o;
    queue.addAll((Collection<?>) o);
    v = (long) coll.size() * RamUsageEstimator.NUM_BYTES_OBJECT_REF;
  } else if (o instanceof Map) {
    final Map<?, ?> map = (Map<?,?>) o;
    queue.addAll(map.keySet());
    queue.addAll(map.values());
    v = 2L * map.size() * RamUsageEstimator.NUM_BYTES_OBJECT_REF;
  } else {
    List<Object> references = new ArrayList<>();
    v = super.accumulateObject(o, shallowSize, fieldValues, references);
    for (Object r : references) {
      // AssertingCodec adds Thread references to make sure objects are consumed in the right thread
      if (r instanceof Thread == false) {
        queue.add(r);
      }
    }
  }
  return v;
}
 
Example 8
Source File: BinaryDocValuesFieldUpdates.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
@Override
public long ramBytesUsed() {
  return super.ramBytesUsed()
    + offsets.ramBytesUsed()
    + lengths.ramBytesUsed()
    + RamUsageEstimator.NUM_BYTES_OBJECT_HEADER
    + 2 * Integer.BYTES
    + 3 * RamUsageEstimator.NUM_BYTES_OBJECT_REF
    + values.bytes().length;
}
 
Example 9
Source File: NumericDocValuesFieldUpdates.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
@Override
public long ramBytesUsed() {
  return values.ramBytesUsed()
      + super.ramBytesUsed()
      + Long.BYTES
      + RamUsageEstimator.NUM_BYTES_OBJECT_REF;
}
 
Example 10
Source File: DocValuesFieldUpdates.java    From lucene-solr with Apache License 2.0 5 votes vote down vote up
@Override
public long ramBytesUsed() {
  return docs.ramBytesUsed()
      + RamUsageEstimator.NUM_BYTES_OBJECT_HEADER
      + 2 * Integer.BYTES
      + 2 + Long.BYTES
      + RamUsageEstimator.NUM_BYTES_OBJECT_REF;
}
 
Example 11
Source File: FieldCache.java    From lucene-solr with Apache License 2.0 4 votes vote down vote up
@Override
public long ramBytesUsed() {
  // don't call on the in-progress value, might make things angry.
  return RamUsageEstimator.NUM_BYTES_OBJECT_REF;
}
 
Example 12
Source File: IndicesRequestCache.java    From Elasticsearch with Apache License 2.0 4 votes vote down vote up
@Override
public long ramBytesUsed() {
    return RamUsageEstimator.NUM_BYTES_OBJECT_REF + RamUsageEstimator.NUM_BYTES_LONG + value.length();
}
 
Example 13
Source File: PagedMutable.java    From lucene-solr with Apache License 2.0 4 votes vote down vote up
@Override
protected long baseRamBytesUsed() {
  return super.baseRamBytesUsed() + RamUsageEstimator.NUM_BYTES_OBJECT_REF;
}
 
Example 14
Source File: FilteredDocIdSet.java    From lucene-solr with Apache License 2.0 4 votes vote down vote up
@Override
public long ramBytesUsed() {
  return RamUsageEstimator.NUM_BYTES_OBJECT_REF + _innerSet.ramBytesUsed();
}
 
Example 15
Source File: SimpleTextDocValuesReader.java    From lucene-solr with Apache License 2.0 4 votes vote down vote up
@Override
public long ramBytesUsed() {
  return BASE_RAM_BYTES_USED + RamUsageEstimator.sizeOf(scratch.bytes())
      + fields.size() * (RamUsageEstimator.NUM_BYTES_OBJECT_REF * 2L + OneField.BASE_RAM_BYTES_USED);
}
 
Example 16
Source File: FSTOrdsOutputs.java    From lucene-solr with Apache License 2.0 4 votes vote down vote up
@Override
public long ramBytesUsed(Output output) {
  return 2 * RamUsageEstimator.NUM_BYTES_OBJECT_HEADER + 2 * Long.BYTES + 2 * RamUsageEstimator.NUM_BYTES_OBJECT_REF + RamUsageEstimator.NUM_BYTES_ARRAY_HEADER + 2 * Integer.BYTES + output.bytes.length;
}
 
Example 17
Source File: LindenFieldCacheImpl.java    From linden with Apache License 2.0 4 votes vote down vote up
@Override
public long ramBytesUsed() {
  // don't call on the in-progress value, might make things angry.
  return RamUsageEstimator.NUM_BYTES_OBJECT_REF;
}
 
Example 18
Source File: VersionValue.java    From Elasticsearch with Apache License 2.0 4 votes vote down vote up
@Override
public long ramBytesUsed() {
    return RamUsageEstimator.NUM_BYTES_OBJECT_HEADER + RamUsageEstimator.NUM_BYTES_LONG + RamUsageEstimator.NUM_BYTES_OBJECT_REF + (translogLocation == null ? 0 : translogLocation.ramBytesUsed());
}
 
Example 19
Source File: SinglePackedOrdinals.java    From Elasticsearch with Apache License 2.0 4 votes vote down vote up
@Override
public long ramBytesUsed() {
    return RamUsageEstimator.NUM_BYTES_OBJECT_REF + reader.ramBytesUsed();
}
 
Example 20
Source File: CardinalityAggregator.java    From Elasticsearch with Apache License 2.0 4 votes vote down vote up
/**
 * Return an approximate memory overhead per bucket for this collector.
 */
public static long memoryOverhead(long maxOrd) {
    return RamUsageEstimator.NUM_BYTES_OBJECT_REF + SHALLOW_FIXEDBITSET_SIZE + (maxOrd + 7) / 8; // 1 bit per ord
}