Java Code Examples for org.apache.calcite.util.ImmutableBitSet#contains()

The following examples show how to use org.apache.calcite.util.ImmutableBitSet#contains() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: RelMdPredicates.java    From Bats with Apache License 2.0 5 votes vote down vote up
/**
 * Infers predicates for an Aggregate.
 *
 * <p>Pulls up predicates that only contains references to columns in the
 * GroupSet. For e.g.
 *
 * <blockquote><pre>
 * inputPullUpExprs : { a &gt; 7, b + c &lt; 10, a + e = 9}
 * groupSet         : { a, b}
 * pulledUpExprs    : { a &gt; 7}
 * </pre></blockquote>
 */
public RelOptPredicateList getPredicates(Aggregate agg, RelMetadataQuery mq) {
  final RelNode input = agg.getInput();
  final RexBuilder rexBuilder = agg.getCluster().getRexBuilder();
  final RelOptPredicateList inputInfo = mq.getPulledUpPredicates(input);
  final List<RexNode> aggPullUpPredicates = new ArrayList<>();

  ImmutableBitSet groupKeys = agg.getGroupSet();
  if (groupKeys.isEmpty()) {
    // "GROUP BY ()" can convert an empty relation to a non-empty relation, so
    // it is not valid to pull up predicates. In particular, consider the
    // predicate "false": it is valid on all input rows (trivially - there are
    // no rows!) but not on the output (there is one row).
    return RelOptPredicateList.EMPTY;
  }
  Mapping m = Mappings.create(MappingType.PARTIAL_FUNCTION,
      input.getRowType().getFieldCount(), agg.getRowType().getFieldCount());

  int i = 0;
  for (int j : groupKeys) {
    m.set(j, i++);
  }

  for (RexNode r : inputInfo.pulledUpPredicates) {
    ImmutableBitSet rCols = RelOptUtil.InputFinder.bits(r);
    if (groupKeys.contains(rCols)) {
      r = r.accept(new RexPermuteInputsShuttle(m, input));
      aggPullUpPredicates.add(r);
    }
  }
  return RelOptPredicateList.of(rexBuilder, aggPullUpPredicates);
}
 
Example 2
Source File: LoptOptimizeJoinRule.java    From calcite with Apache License 2.0 5 votes vote down vote up
/**
 * Locates from a list of filters those that correspond to a particular join
 * tree. Then, for each of those filters, extracts the fields corresponding
 * to a particular factor, setting them in a bitmap.
 *
 * @param multiJoin join factors being optimized
 * @param filters list of join filters
 * @param joinFactors bitmap containing the factors in a particular join
 * tree
 * @param factorStart the initial offset of the factor whose join keys will
 * be extracted
 * @param nFields the number of fields in the factor whose join keys will be
 * extracted
 * @param joinKeys the bitmap that will be set with the join keys
 */
private void setFactorJoinKeys(
    LoptMultiJoin multiJoin,
    List<RexNode> filters,
    ImmutableBitSet joinFactors,
    int factorStart,
    int nFields,
    ImmutableBitSet.Builder joinKeys) {
  for (RexNode joinFilter : filters) {
    ImmutableBitSet filterFactors =
        multiJoin.getFactorsRefByJoinFilter(joinFilter);

    // if all factors in the join filter are in the bitmap containing
    // the factors in a join tree, then from that filter, add the
    // fields corresponding to the specified factor to the join key
    // bitmap; in doing so, adjust the join keys so they start at
    // offset 0
    if (joinFactors.contains(filterFactors)) {
      ImmutableBitSet joinFields =
          multiJoin.getFieldsRefByJoinFilter(joinFilter);
      for (int field = joinFields.nextSetBit(factorStart);
           (field >= 0)
               && (field < (factorStart + nFields));
           field = joinFields.nextSetBit(field + 1)) {
        joinKeys.set(field - factorStart);
      }
    }
  }
}
 
Example 3
Source File: SimpleProfiler.java    From calcite with Apache License 2.0 5 votes vote down vote up
/** Returns whether a set of column ordinals
 * matches or contains a unique key.
 * If {@code strict}, it must contain a unique key. */
private boolean containsKey(ImmutableBitSet ordinals, boolean strict) {
  for (ImmutableBitSet keyOrdinals : keyOrdinalLists) {
    if (ordinals.contains(keyOrdinals)) {
      return !(strict && keyOrdinals.equals(ordinals));
    }
  }
  return false;
}
 
Example 4
Source File: RelMdPredicates.java    From calcite with Apache License 2.0 5 votes vote down vote up
/**
 * Infers predicates for an Aggregate.
 *
 * <p>Pulls up predicates that only contains references to columns in the
 * GroupSet. For e.g.
 *
 * <blockquote><pre>
 * inputPullUpExprs : { a &gt; 7, b + c &lt; 10, a + e = 9}
 * groupSet         : { a, b}
 * pulledUpExprs    : { a &gt; 7}
 * </pre></blockquote>
 */
public RelOptPredicateList getPredicates(Aggregate agg, RelMetadataQuery mq) {
  final RelNode input = agg.getInput();
  final RexBuilder rexBuilder = agg.getCluster().getRexBuilder();
  final RelOptPredicateList inputInfo = mq.getPulledUpPredicates(input);
  final List<RexNode> aggPullUpPredicates = new ArrayList<>();

  ImmutableBitSet groupKeys = agg.getGroupSet();
  if (groupKeys.isEmpty()) {
    // "GROUP BY ()" can convert an empty relation to a non-empty relation, so
    // it is not valid to pull up predicates. In particular, consider the
    // predicate "false": it is valid on all input rows (trivially - there are
    // no rows!) but not on the output (there is one row).
    return RelOptPredicateList.EMPTY;
  }
  Mapping m = Mappings.create(MappingType.PARTIAL_FUNCTION,
      input.getRowType().getFieldCount(), agg.getRowType().getFieldCount());

  int i = 0;
  for (int j : groupKeys) {
    m.set(j, i++);
  }

  for (RexNode r : inputInfo.pulledUpPredicates) {
    ImmutableBitSet rCols = RelOptUtil.InputFinder.bits(r);
    if (groupKeys.contains(rCols)) {
      r = r.accept(new RexPermuteInputsShuttle(m, input));
      aggPullUpPredicates.add(r);
    }
  }
  return RelOptPredicateList.of(rexBuilder, aggPullUpPredicates);
}
 
Example 5
Source File: Statistics.java    From Bats with Apache License 2.0 5 votes vote down vote up
/** Returns a statistic with a given row count, set of unique keys,
 * referential constraints, and collations. */
public static Statistic of(final Double rowCount,
    final List<ImmutableBitSet> keys,
    final List<RelReferentialConstraint> referentialConstraints,
    final List<RelCollation> collations) {
  return new Statistic() {
    public Double getRowCount() {
      return rowCount;
    }

    public boolean isKey(ImmutableBitSet columns) {
      for (ImmutableBitSet key : keys) {
        if (columns.contains(key)) {
          return true;
        }
      }
      return false;
    }

    public List<RelReferentialConstraint> getReferentialConstraints() {
      return referentialConstraints;
    }

    public List<RelCollation> getCollations() {
      return collations;
    }

    public RelDistribution getDistribution() {
      return RelDistributionTraitDef.INSTANCE.getDefault();
    }
  };
}
 
Example 6
Source File: RelOptUtil.java    From Bats with Apache License 2.0 5 votes vote down vote up
/**
 * Splits a filter into two lists, depending on whether or not the filter
 * only references its child input
 *
 * @param childBitmap Fields in the child
 * @param predicate   filters that will be split
 * @param pushable    returns the list of filters that can be pushed to the
 *                    child input
 * @param notPushable returns the list of filters that cannot be pushed to
 *                    the child input
 */
public static void splitFilters(ImmutableBitSet childBitmap, RexNode predicate, List<RexNode> pushable,
        List<RexNode> notPushable) {
    // for each filter, if the filter only references the child inputs,
    // then it can be pushed
    for (RexNode filter : conjunctions(predicate)) {
        ImmutableBitSet filterRefs = InputFinder.bits(filter);
        if (childBitmap.contains(filterRefs)) {
            pushable.add(filter);
        } else {
            notPushable.add(filter);
        }
    }
}
 
Example 7
Source File: RelMetadataTest.java    From calcite with Apache License 2.0 5 votes vote down vote up
/** Returns whether {@code keys} is unique, that is, whether it or a superset
 * is in {@code keySets}. */
private boolean isUnique(Set<ImmutableBitSet> uniqueKeys, ImmutableBitSet key) {
  for (ImmutableBitSet uniqueKey : uniqueKeys) {
    if (key.contains(uniqueKey)) {
      return true;
    }
  }
  return false;
}
 
Example 8
Source File: QuarkCube.java    From quark with Apache License 2.0 5 votes vote down vote up
private void validateCubeLatticeFilter(Lattice.Builder latticeBuilder) {
  if (latticeBuilder.filter != null) {
    ImmutableBitSet rCols = RelOptUtil.InputFinder.bits(latticeBuilder.filter);
    Set<Integer> dimSet = new HashSet<>();
    for (Dimension dimension : dimensions) {
      dimSet.add(latticeBuilder.resolveColumn(dimension.qualifiedCol).ordinal);
    }
    ImmutableBitSet dims = ImmutableBitSet.of(dimSet);
    if (!dims.contains(rCols)) {
      throw new RuntimeException("Cube filter is only allowed on dimensions");
    }
  }
}
 
Example 9
Source File: RelMdColumnUniqueness.java    From calcite with Apache License 2.0 5 votes vote down vote up
public Boolean areColumnsUnique(Aggregate rel, RelMetadataQuery mq,
    ImmutableBitSet columns, boolean ignoreNulls) {
  columns = decorateWithConstantColumnsFromPredicates(columns, rel, mq);
  // group by keys form a unique key
  ImmutableBitSet groupKey = ImmutableBitSet.range(rel.getGroupCount());
  return columns.contains(groupKey);
}
 
Example 10
Source File: LoptSemiJoinOptimizer.java    From calcite with Apache License 2.0 4 votes vote down vote up
/**
 * Determines whether a join of the dimension table in a semijoin can be
 * removed. It can be if the dimension keys are unique and the only fields
 * referenced from the dimension table are its semijoin keys. The semijoin
 * keys can be mapped to the corresponding keys from the fact table (because
 * of the equality condition associated with the semijoin keys). Therefore,
 * that's why the dimension table can be removed even though those fields
 * are referenced elsewhere in the query tree.
 *
 * @param multiJoin join factors being optimized
 * @param semiJoin semijoin under consideration
 * @param factIdx id of the fact table in the semijoin
 * @param dimIdx id of the dimension table in the semijoin
 */
private void removeJoin(
    LoptMultiJoin multiJoin,
    LogicalJoin semiJoin,
    int factIdx,
    int dimIdx) {
  // if the dimension can be removed because of another semijoin, then
  // no need to proceed any further
  if (multiJoin.getJoinRemovalFactor(dimIdx) != null) {
    return;
  }

  // Check if the semijoin keys corresponding to the dimension table
  // are unique.  The semijoin will filter out the nulls.
  final ImmutableBitSet dimKeys = ImmutableBitSet.of(semiJoin.analyzeCondition().rightKeys);
  final RelNode dimRel = multiJoin.getJoinFactor(dimIdx);
  if (!RelMdUtil.areColumnsDefinitelyUniqueWhenNullsFiltered(mq, dimRel,
      dimKeys)) {
    return;
  }

  // check that the only fields referenced from the dimension table
  // in either its projection or join conditions are the dimension
  // keys
  ImmutableBitSet dimProjRefs = multiJoin.getProjFields(dimIdx);
  if (dimProjRefs == null) {
    int nDimFields = multiJoin.getNumFieldsInJoinFactor(dimIdx);
    dimProjRefs = ImmutableBitSet.range(0, nDimFields);
  }
  if (!dimKeys.contains(dimProjRefs)) {
    return;
  }
  int [] dimJoinRefCounts = multiJoin.getJoinFieldRefCounts(dimIdx);
  for (int i = 0; i < dimJoinRefCounts.length; i++) {
    if (dimJoinRefCounts[i] > 0) {
      if (!dimKeys.get(i)) {
        return;
      }
    }
  }

  // criteria met; keep track of the fact table and the semijoin that
  // allow the join of this dimension table to be removed
  multiJoin.setJoinRemovalFactor(dimIdx, factIdx);
  multiJoin.setJoinRemovalSemiJoin(dimIdx, semiJoin);

  // if the dimension table doesn't reference anything in its projection
  // and the only fields referenced in its joins are the dimension keys
  // of this semijoin, then we can decrement the join reference counts
  // corresponding to the fact table's semijoin keys, since the
  // dimension table doesn't need to use those keys
  if (dimProjRefs.cardinality() != 0) {
    return;
  }
  for (int i = 0; i < dimJoinRefCounts.length; i++) {
    if (dimJoinRefCounts[i] > 1) {
      return;
    } else if (dimJoinRefCounts[i] == 1) {
      if (!dimKeys.get(i)) {
        return;
      }
    }
  }
  int [] factJoinRefCounts = multiJoin.getJoinFieldRefCounts(factIdx);
  for (Integer key : semiJoin.analyzeCondition().leftKeys) {
    factJoinRefCounts[key]--;
  }
}
 
Example 11
Source File: RelMdColumnUniqueness.java    From Bats with Apache License 2.0 4 votes vote down vote up
public Boolean areColumnsUnique(Aggregate rel, RelMetadataQuery mq,
    ImmutableBitSet columns, boolean ignoreNulls) {
  // group by keys form a unique key
  ImmutableBitSet groupKey = ImmutableBitSet.range(rel.getGroupCount());
  return columns.contains(groupKey);
}
 
Example 12
Source File: LoptOptimizeJoinRule.java    From calcite with Apache License 2.0 4 votes vote down vote up
/**
 * Determines which join filters can be added to the current join tree. Note
 * that the join filter still reflects the original join ordering. It will
 * only be adjusted to reflect the new join ordering if the "adjust"
 * parameter is set to true.
 *
 * @param multiJoin join factors being optimized
 * @param leftTree left subtree of the join tree
 * @param leftIdx if &ge; 0, only consider filters that reference leftIdx in
 * leftTree; otherwise, consider all filters that reference any factor in
 * leftTree
 * @param rightTree right subtree of the join tree
 * @param filtersToAdd remaining join filters that need to be added; those
 * that are added are removed from the list
 * @param adjust if true, adjust filter to reflect new join ordering
 *
 * @return AND'd expression of the join filters that can be added to the
 * current join tree
 */
private RexNode addFilters(
    LoptMultiJoin multiJoin,
    LoptJoinTree leftTree,
    int leftIdx,
    LoptJoinTree rightTree,
    List<RexNode> filtersToAdd,
    boolean adjust) {
  // loop through the remaining filters to be added and pick out the
  // ones that reference only the factors in the new join tree
  final RexBuilder rexBuilder =
      multiJoin.getMultiJoinRel().getCluster().getRexBuilder();
  final ImmutableBitSet.Builder childFactorBuilder =
      ImmutableBitSet.builder();
  childFactorBuilder.addAll(rightTree.getTreeOrder());
  if (leftIdx >= 0) {
    childFactorBuilder.set(leftIdx);
  } else {
    childFactorBuilder.addAll(leftTree.getTreeOrder());
  }
  for (int child : rightTree.getTreeOrder()) {
    childFactorBuilder.set(child);
  }

  final ImmutableBitSet childFactor = childFactorBuilder.build();
  RexNode condition = null;
  final ListIterator<RexNode> filterIter = filtersToAdd.listIterator();
  while (filterIter.hasNext()) {
    RexNode joinFilter = filterIter.next();
    ImmutableBitSet filterBitmap =
        multiJoin.getFactorsRefByJoinFilter(joinFilter);

    // if all factors in the join filter are in the join tree,
    // AND the filter to the current join condition
    if (childFactor.contains(filterBitmap)) {
      if (condition == null) {
        condition = joinFilter;
      } else {
        condition =
            rexBuilder.makeCall(
                SqlStdOperatorTable.AND,
                condition,
                joinFilter);
      }
      filterIter.remove();
    }
  }

  if (adjust && (condition != null)) {
    int [] adjustments = new int[multiJoin.getNumTotalFields()];
    if (needsAdjustment(
        multiJoin,
        adjustments,
        leftTree,
        rightTree,
        false)) {
      condition =
          condition.accept(
              new RelOptUtil.RexInputConverter(
                  rexBuilder,
                  multiJoin.getMultiJoinFields(),
                  leftTree.getJoinTree().getRowType().getFieldList(),
                  rightTree.getJoinTree().getRowType().getFieldList(),
                  adjustments));
    }
  }

  if (condition == null) {
    condition = rexBuilder.makeLiteral(true);
  }

  return condition;
}
 
Example 13
Source File: RelMdPredicates.java    From calcite with Apache License 2.0 4 votes vote down vote up
private boolean checkTarget(ImmutableBitSet inferringFields,
    Set<RexNode> allExprs, RexNode tr) {
  return inferringFields.contains(RelOptUtil.InputFinder.bits(tr))
      && !allExprs.contains(tr)
      && !isAlwaysTrue(tr);
}
 
Example 14
Source File: RelMdPredicates.java    From calcite with Apache License 2.0 4 votes vote down vote up
/** Converts a predicate on a particular set of columns into a predicate on
 * a subset of those columns, weakening if necessary.
 *
 * <p>If not possible to simplify, returns {@code true}, which is the weakest
 * possible predicate.
 *
 * <p>Examples:<ol>
 * <li>The predicate {@code $7 = $9} on columns [7]
 *     becomes {@code $7 is not null}
 * <li>The predicate {@code $7 = $9 + $11} on columns [7, 9]
 *     becomes {@code $7 is not null or $9 is not null}
 * <li>The predicate {@code $7 = $9 and $9 = 5} on columns [7] becomes
 *   {@code $7 = 5}
 * <li>The predicate
 *   {@code $7 = $9 and ($9 = $1 or $9 = $2) and $1 > 3 and $2 > 10}
 *   on columns [7] becomes {@code $7 > 3}
 * </ol>
 *
 * <p>We currently only handle examples 1 and 2.
 *
 * @param rexBuilder Rex builder
 * @param input Input relational expression
 * @param r Predicate expression
 * @param columnsMapped Columns which the final predicate can reference
 * @return Predicate expression narrowed to reference only certain columns
 */
private RexNode projectPredicate(final RexBuilder rexBuilder, RelNode input,
    RexNode r, ImmutableBitSet columnsMapped) {
  ImmutableBitSet rCols = RelOptUtil.InputFinder.bits(r);
  if (columnsMapped.contains(rCols)) {
    // All required columns are present. No need to weaken.
    return r;
  }
  if (columnsMapped.intersects(rCols)) {
    final List<RexNode> list = new ArrayList<>();
    for (int c : columnsMapped.intersect(rCols)) {
      if (input.getRowType().getFieldList().get(c).getType().isNullable()
          && Strong.isNull(r, ImmutableBitSet.of(c))) {
        list.add(
            rexBuilder.makeCall(SqlStdOperatorTable.IS_NOT_NULL,
                rexBuilder.makeInputRef(input, c)));
      }
    }
    if (!list.isEmpty()) {
      return RexUtil.composeDisjunction(rexBuilder, list);
    }
  }
  // Cannot weaken to anything non-trivial
  return rexBuilder.makeLiteral(true);
}
 
Example 15
Source File: RelMdUniqueKeys.java    From calcite with Apache License 2.0 4 votes vote down vote up
private Set<ImmutableBitSet> getProjectUniqueKeys(SingleRel rel, RelMetadataQuery mq,
    boolean ignoreNulls, List<RexNode> projExprs) {
  // LogicalProject maps a set of rows to a different set;
  // Without knowledge of the mapping function(whether it
  // preserves uniqueness), it is only safe to derive uniqueness
  // info from the child of a project when the mapping is f(a) => a.
  //
  // Further more, the unique bitset coming from the child needs
  // to be mapped to match the output of the project.

  // Single input can be mapped to multiple outputs
  ImmutableMultimap.Builder<Integer, Integer> inToOutPosBuilder = ImmutableMultimap.builder();
  ImmutableBitSet.Builder mappedInColumnsBuilder = ImmutableBitSet.builder();

  // Build an input to output position map.
  for (int i = 0; i < projExprs.size(); i++) {
    RexNode projExpr = projExprs.get(i);
    if (projExpr instanceof RexInputRef) {
      int inputIndex = ((RexInputRef) projExpr).getIndex();
      inToOutPosBuilder.put(inputIndex, i);
      mappedInColumnsBuilder.set(inputIndex);
    }
  }
  ImmutableBitSet inColumnsUsed = mappedInColumnsBuilder.build();

  if (inColumnsUsed.isEmpty()) {
    // if there's no RexInputRef in the projected expressions
    // return empty set.
    return ImmutableSet.of();
  }

  Set<ImmutableBitSet> childUniqueKeySet =
      mq.getUniqueKeys(rel.getInput(), ignoreNulls);

  if (childUniqueKeySet == null) {
    return ImmutableSet.of();
  }

  Map<Integer, ImmutableBitSet> mapInToOutPos =
      Maps.transformValues(inToOutPosBuilder.build().asMap(), ImmutableBitSet::of);

  ImmutableSet.Builder<ImmutableBitSet> resultBuilder = ImmutableSet.builder();
  // Now add to the projUniqueKeySet the child keys that are fully
  // projected.
  for (ImmutableBitSet colMask : childUniqueKeySet) {
    ImmutableBitSet.Builder tmpMask = ImmutableBitSet.builder();
    if (!inColumnsUsed.contains(colMask)) {
      // colMask contains a column that is not projected as RexInput => the key is not unique
      continue;
    }
    // colMask is mapped to output project, however, the column can be mapped more than once:
    // select id, id, id, unique2, unique2
    // the resulting unique keys would be {{0},{3}}, {{0},{4}}, {{0},{1},{4}}, ...

    Iterable<List<ImmutableBitSet>> product = Linq4j.product(
        Iterables.transform(colMask,
            in -> Iterables.filter(mapInToOutPos.get(in).powerSet(), bs -> !bs.isEmpty())));

    resultBuilder.addAll(Iterables.transform(product, ImmutableBitSet::union));
  }
  return resultBuilder.build();
}
 
Example 16
Source File: MockCatalogReader.java    From calcite with Apache License 2.0 4 votes vote down vote up
public boolean isKey(ImmutableBitSet columns) {
  return !keyList.isEmpty()
      && columns.contains(ImmutableBitSet.of(keyList));
}
 
Example 17
Source File: LoptOptimizeJoinRule.java    From Bats with Apache License 2.0 4 votes vote down vote up
/**
 * Determines which join filters can be added to the current join tree. Note
 * that the join filter still reflects the original join ordering. It will
 * only be adjusted to reflect the new join ordering if the "adjust"
 * parameter is set to true.
 *
 * @param multiJoin join factors being optimized
 * @param leftTree left subtree of the join tree
 * @param leftIdx if &ge; 0, only consider filters that reference leftIdx in
 * leftTree; otherwise, consider all filters that reference any factor in
 * leftTree
 * @param rightTree right subtree of the join tree
 * @param filtersToAdd remaining join filters that need to be added; those
 * that are added are removed from the list
 * @param adjust if true, adjust filter to reflect new join ordering
 *
 * @return AND'd expression of the join filters that can be added to the
 * current join tree
 */
private RexNode addFilters(
    LoptMultiJoin multiJoin,
    LoptJoinTree leftTree,
    int leftIdx,
    LoptJoinTree rightTree,
    List<RexNode> filtersToAdd,
    boolean adjust) {
  // loop through the remaining filters to be added and pick out the
  // ones that reference only the factors in the new join tree
  final RexBuilder rexBuilder =
      multiJoin.getMultiJoinRel().getCluster().getRexBuilder();
  final ImmutableBitSet.Builder childFactorBuilder =
      ImmutableBitSet.builder();
  childFactorBuilder.addAll(rightTree.getTreeOrder());
  if (leftIdx >= 0) {
    childFactorBuilder.set(leftIdx);
  } else {
    childFactorBuilder.addAll(leftTree.getTreeOrder());
  }
  for (int child : rightTree.getTreeOrder()) {
    childFactorBuilder.set(child);
  }

  final ImmutableBitSet childFactor = childFactorBuilder.build();
  RexNode condition = null;
  final ListIterator<RexNode> filterIter = filtersToAdd.listIterator();
  while (filterIter.hasNext()) {
    RexNode joinFilter = filterIter.next();
    ImmutableBitSet filterBitmap =
        multiJoin.getFactorsRefByJoinFilter(joinFilter);

    // if all factors in the join filter are in the join tree,
    // AND the filter to the current join condition
    if (childFactor.contains(filterBitmap)) {
      if (condition == null) {
        condition = joinFilter;
      } else {
        condition =
            rexBuilder.makeCall(
                SqlStdOperatorTable.AND,
                condition,
                joinFilter);
      }
      filterIter.remove();
    }
  }

  if (adjust && (condition != null)) {
    int [] adjustments = new int[multiJoin.getNumTotalFields()];
    if (needsAdjustment(
        multiJoin,
        adjustments,
        leftTree,
        rightTree,
        false)) {
      condition =
          condition.accept(
              new RelOptUtil.RexInputConverter(
                  rexBuilder,
                  multiJoin.getMultiJoinFields(),
                  leftTree.getJoinTree().getRowType().getFieldList(),
                  rightTree.getJoinTree().getRowType().getFieldList(),
                  adjustments));
    }
  }

  if (condition == null) {
    condition = rexBuilder.makeLiteral(true);
  }

  return condition;
}
 
Example 18
Source File: AggregateMergeRule.java    From calcite with Apache License 2.0 4 votes vote down vote up
public void onMatch(RelOptRuleCall call) {
  final Aggregate topAgg = call.rel(0);
  final Aggregate bottomAgg = call.rel(1);
  if (topAgg.getGroupCount() > bottomAgg.getGroupCount()) {
    return;
  }

  final ImmutableBitSet bottomGroupSet = bottomAgg.getGroupSet();
  final Map<Integer, Integer> map = new HashMap<>();
  bottomGroupSet.forEach(v -> map.put(map.size(), v));
  for (int k : topAgg.getGroupSet()) {
    if (!map.containsKey(k)) {
      return;
    }
  }

  // top aggregate keys must be subset of lower aggregate keys
  final ImmutableBitSet topGroupSet = topAgg.getGroupSet().permute(map);
  if (!bottomGroupSet.contains(topGroupSet)) {
    return;
  }

  boolean hasEmptyGroup = topAgg.getGroupSets()
      .stream().anyMatch(n -> n.isEmpty());

  final List<AggregateCall> finalCalls = new ArrayList<>();
  for (AggregateCall topCall : topAgg.getAggCallList()) {
    if (!isAggregateSupported(topCall)
        || topCall.getArgList().size() == 0) {
      return;
    }
    // Make sure top aggregate argument refers to one of the aggregate
    int bottomIndex = topCall.getArgList().get(0) - bottomGroupSet.cardinality();
    if (bottomIndex >= bottomAgg.getAggCallList().size()
        || bottomIndex < 0) {
      return;
    }
    AggregateCall bottomCall = bottomAgg.getAggCallList().get(bottomIndex);
    // Should not merge if top agg with empty group keys and the lower agg
    // function is COUNT, because in case of empty input for lower agg,
    // the result is empty, if we merge them, we end up with 1 result with
    // 0, which is wrong.
    if (!isAggregateSupported(bottomCall)
        || (bottomCall.getAggregation() == SqlStdOperatorTable.COUNT
             && hasEmptyGroup)) {
      return;
    }
    SqlSplittableAggFunction splitter = Objects.requireNonNull(
        bottomCall.getAggregation().unwrap(SqlSplittableAggFunction.class));
    AggregateCall finalCall = splitter.merge(topCall, bottomCall);
    // fail to merge the aggregate call, bail out
    if (finalCall == null) {
      return;
    }
    finalCalls.add(finalCall);
  }

  // re-map grouping sets
  ImmutableList<ImmutableBitSet> newGroupingSets = null;
  if (topAgg.getGroupType() != Group.SIMPLE) {
    newGroupingSets =
        ImmutableBitSet.ORDERING.immutableSortedCopy(
            ImmutableBitSet.permute(topAgg.getGroupSets(), map));
  }

  final Aggregate finalAgg =
      topAgg.copy(topAgg.getTraitSet(), bottomAgg.getInput(), topGroupSet,
          newGroupingSets, finalCalls);
  call.transformTo(finalAgg);
}
 
Example 19
Source File: RelMdPredicates.java    From Bats with Apache License 2.0 4 votes vote down vote up
private boolean checkTarget(ImmutableBitSet inferringFields,
    Set<RexNode> allExprs, RexNode tr) {
  return inferringFields.contains(RelOptUtil.InputFinder.bits(tr))
      && !allExprs.contains(tr)
      && !isAlwaysTrue(tr);
}
 
Example 20
Source File: RelMdPredicates.java    From Bats with Apache License 2.0 4 votes vote down vote up
/** Converts a predicate on a particular set of columns into a predicate on
 * a subset of those columns, weakening if necessary.
 *
 * <p>If not possible to simplify, returns {@code true}, which is the weakest
 * possible predicate.
 *
 * <p>Examples:<ol>
 * <li>The predicate {@code $7 = $9} on columns [7]
 *     becomes {@code $7 is not null}
 * <li>The predicate {@code $7 = $9 + $11} on columns [7, 9]
 *     becomes {@code $7 is not null or $9 is not null}
 * <li>The predicate {@code $7 = $9 and $9 = 5} on columns [7] becomes
 *   {@code $7 = 5}
 * <li>The predicate
 *   {@code $7 = $9 and ($9 = $1 or $9 = $2) and $1 > 3 and $2 > 10}
 *   on columns [7] becomes {@code $7 > 3}
 * </ol>
 *
 * <p>We currently only handle examples 1 and 2.
 *
 * @param rexBuilder Rex builder
 * @param input Input relational expression
 * @param r Predicate expression
 * @param columnsMapped Columns which the final predicate can reference
 * @return Predicate expression narrowed to reference only certain columns
 */
private RexNode projectPredicate(final RexBuilder rexBuilder, RelNode input,
    RexNode r, ImmutableBitSet columnsMapped) {
  ImmutableBitSet rCols = RelOptUtil.InputFinder.bits(r);
  if (columnsMapped.contains(rCols)) {
    // All required columns are present. No need to weaken.
    return r;
  }
  if (columnsMapped.intersects(rCols)) {
    final List<RexNode> list = new ArrayList<>();
    for (int c : columnsMapped.intersect(rCols)) {
      if (input.getRowType().getFieldList().get(c).getType().isNullable()
          && Strong.isNull(r, ImmutableBitSet.of(c))) {
        list.add(
            rexBuilder.makeCall(SqlStdOperatorTable.IS_NOT_NULL,
                rexBuilder.makeInputRef(input, c)));
      }
    }
    if (!list.isEmpty()) {
      return RexUtil.composeDisjunction(rexBuilder, list);
    }
  }
  // Cannot weaken to anything non-trivial
  return rexBuilder.makeLiteral(true);
}