Java Code Examples for com.google.common.collect.Iterators#mergeSorted()

The following examples show how to use com.google.common.collect.Iterators#mergeSorted() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: EmptyInstance.java    From heroic with Apache License 2.0 6 votes vote down vote up
private <T extends Metric> AggregationOutput collectGroup(
    final Map<String, String> key, final SessionPair<T> collected,
    final Function<List<T>, MetricCollection> builder
) {
    final ImmutableList.Builder<List<T>> iterables = ImmutableList.builder();

    for (final List<T> d : collected.data) {
        iterables.add(d);
    }

    final Set<Series> series = ImmutableSet.copyOf(Iterables.concat(collected.series));

    /* no need to merge, single results are already sorted */
    if (collected.data.size() == 1) {
        return new AggregationOutput(key, series,
            builder.apply(iterables.build().iterator().next()));
    }

    final ImmutableList<Iterator<T>> iterators =
        ImmutableList.copyOf(iterables.build().stream().map(Iterable::iterator).iterator());
    final Iterator<T> metrics = Iterators.mergeSorted(iterators, Metric.comparator);

    return new AggregationOutput(key, series, builder.apply(ImmutableList.copyOf(metrics)));
}
 
Example 2
Source File: LogReader.java    From vespa with Apache License 2.0 5 votes vote down vote up
void writeLogs(OutputStream out, Instant from, Instant to) {
    double fromSeconds = from.getEpochSecond() + from.getNano() / 1e9;
    double toSeconds = to.getEpochSecond() + to.getNano() / 1e9;
    BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(out));
    try {
        for (List<Path> logs : getMatchingFiles(from, to)) {
            List<LogLineIterator> logLineIterators = new ArrayList<>();
            try {
                // Logs in each sub-list contain entries covering the same time interval, so do a merge sort while reading
                for (Path log : logs)
                    logLineIterators.add(new LogLineIterator(log, fromSeconds, toSeconds));

                Iterator<LineWithTimestamp> lines = Iterators.mergeSorted(logLineIterators,
                                                                          Comparator.comparingDouble(LineWithTimestamp::timestamp));
                while (lines.hasNext()) {
                    writer.write(lines.next().line());
                    writer.newLine();
                }
            }
            catch (IOException e) {
                throw new UncheckedIOException(e);
            }
            finally {
                for (LogLineIterator ll : logLineIterators) {
                    try { ll.close(); } catch (IOException ignored) { }
                }
            }
        }
    }
    finally {
        Exceptions.uncheck(writer::flush);
    }
}
 
Example 3
Source File: HMMSegmentProcessor.java    From gatk-protected with BSD 3-Clause "New" or "Revised" License 5 votes vote down vote up
/**
 * Create a merging iterator that sorts all the segments across samples by
 * their coordinates.
 * <p>
 *     The order between contigs is determined by the input target collection.
 * </p>
 * <p>
 *     The input map of sample name to segment list is assumed to contain all
 *     segments within each value list already in the correct order.
 * </p>
 *
 * @param masterTargetCollection the target collection that encompasses targets from all samples
 * @param allSegments map with all segment lists coming from different samples.
 * @return never null.
 */
private Iterator<HiddenStateSegmentRecord<STATE, TARGET>> composeTargetSortedSegmentRecordIterator(
        final TargetCollection<TARGET> masterTargetCollection,
        final Map<String, List<HiddenStateSegment<STATE, TARGET>>> allSegments) {
    final List<Iterator<HiddenStateSegmentRecord<STATE, TARGET>>> allSegmentRecordIterators =
            allSegments.entrySet().stream()
                    .map(e -> e.getValue().stream()
                            .map(s -> new HiddenStateSegmentRecord<>(e.getKey(), s))
                            .iterator())
                    .collect(Collectors.toList());

    final Comparator<HiddenStateSegmentRecord<STATE, TARGET>> recordComparator = (o1, o2) -> {
        final HiddenStateSegment<STATE, TARGET> s1 = o1.getSegment();
        final HiddenStateSegment<STATE, TARGET> s2 = o2.getSegment();
        // Using the index-range.from we make sure we sort first by the contig over
        // the start position in the same order as contigs are present in
        // the input data or target collection.
        final IndexRange ir1 = masterTargetCollection.indexRange(o1.getSegment());
        final IndexRange ir2 = masterTargetCollection.indexRange(o2.getSegment());

        final int fromCmp = Integer.compare(ir1.from, ir2.from);
        if (fromCmp != 0) {
            return fromCmp;
        }
        // if fromCmp == 0, they must be in the same contig,
        // then we can sort based on segment start and then end.
        final int startCmp = Integer.compare(s1.getStart(), s2.getStart());
        if (startCmp != 0) {
            return startCmp;
        }
        return Integer.compare(s1.getEnd(), s2.getEnd());
    };

    return Iterators.mergeSorted(allSegmentRecordIterators, recordComparator);
}
 
Example 4
Source File: HMMPostProcessor.java    From gatk-protected with BSD 3-Clause "New" or "Revised" License 5 votes vote down vote up
/**
 * Create a merging iterator that sorts all the segments across samples by
 * their coordinates.
 * <p>
 *     The order between contigs is determined by the input target collection.
 * </p>
 * <p>
 *     The input map of sample name to segment list is assumed to contain all
 *     segments within each value list already in the correct order.
 * </p>
 *
 * @param masterTargetCollection the target collection that encompasses targets from all samples
 * @param allSegments map with all segment lists coming from different samples.
 * @return never null.
 */
private Iterator<HiddenStateSegmentRecord<S, T>> composeTargetSortedSegmentRecordIterator(
        final TargetCollection<T> masterTargetCollection,
        final Map<String, List<HiddenStateSegment<S, T>>> allSegments) {
    final List<Iterator<HiddenStateSegmentRecord<S, T>>> allSegmentRecordIterators =
            allSegments.entrySet().stream()
                    .map(e -> e.getValue().stream()
                            .map(s -> new HiddenStateSegmentRecord<>(e.getKey(), s))
                            .iterator())
                    .collect(Collectors.toList());

    final Comparator<HiddenStateSegmentRecord<S, T>> recordComparator = (o1, o2) -> {
        final HiddenStateSegment<S, T> s1 = o1.getSegment();
        final HiddenStateSegment<S, T> s2 = o2.getSegment();
        // Using the index-range.from we make sure we sort first by the contig over
        // the start position in the same order as contigs are present in
        // the input data or target collection.
        final IndexRange ir1 = masterTargetCollection.indexRange(o1.getSegment());
        final IndexRange ir2 = masterTargetCollection.indexRange(o2.getSegment());

        final int fromCmp = Integer.compare(ir1.from, ir2.from);
        if (fromCmp != 0) {
            return fromCmp;
        }
        // if fromCmp == 0, they must be in the same contig,
        // then we can sort based on segment start and then end.
        final int startCmp = Integer.compare(s1.getStart(), s2.getStart());
        if (startCmp != 0) {
            return startCmp;
        }
        return Integer.compare(s1.getEnd(), s2.getEnd());
    };

    return Iterators.mergeSorted(allSegmentRecordIterators, recordComparator);
}