org.apache.hadoop.fs.GlobPattern Java Examples

The following examples show how to use org.apache.hadoop.fs.GlobPattern. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: Name.java    From hadoop with Apache License 2.0 5 votes vote down vote up
@Override
public void prepare() throws IOException {
  String argPattern = getArgument(1);
  if (!caseSensitive) {
    argPattern = StringUtils.toLowerCase(argPattern);
  }
  globPattern = new GlobPattern(argPattern);
}
 
Example #2
Source File: Name.java    From big-c with Apache License 2.0 5 votes vote down vote up
@Override
public void prepare() throws IOException {
  String argPattern = getArgument(1);
  if (!caseSensitive) {
    argPattern = StringUtils.toLowerCase(argPattern);
  }
  globPattern = new GlobPattern(argPattern);
}
 
Example #3
Source File: FileSystemUtils.java    From Cubert with Apache License 2.0 5 votes vote down vote up
public static Path getFirstMatch(FileSystem fs, Path path, String globPatternStr, boolean recursive)
    throws IOException
{
    RemoteIterator<LocatedFileStatus> files = fs.listFiles(path, recursive);
    GlobPattern globPattern = new GlobPattern(globPatternStr);

    while (files.hasNext())
    {
        Path aFile = files.next().getPath();
        if(globPattern.matches(aFile.getName()))
            return aFile;
    }

    return null;
}
 
Example #4
Source File: FSDatasetDescriptor.java    From incubator-gobblin with Apache License 2.0 5 votes vote down vote up
/**
 * A helper to determine if the path description of this {@link DatasetDescriptor} is a superset of paths
 * accepted by the other {@link DatasetDescriptor}. If the path description of the other {@link DatasetDescriptor}
 * is a glob pattern, we return false.
 *
 * @param otherPath a glob pattern that describes a set of paths.
 * @return true if the glob pattern described by the otherPath matches the path in this {@link DatasetDescriptor}.
 */
private boolean isPathContaining(String otherPath) {
  if (otherPath == null) {
    return false;
  }
  if (DatasetDescriptorConfigKeys.DATASET_DESCRIPTOR_CONFIG_ANY.equals(this.getPath())) {
    return true;
  }
  if (PathUtils.isGlob(new Path(otherPath))) {
    return false;
  }
  GlobPattern globPattern = new GlobPattern(this.getPath());
  return globPattern.matches(otherPath);
}
 
Example #5
Source File: HiveDatasetDescriptor.java    From incubator-gobblin with Apache License 2.0 5 votes vote down vote up
String createHiveDatasetWhitelist() {
  if (new GlobPattern(this.databaseName).hasWildcard()) {
    return this.databaseName + ".*";
  } else {
    return this.databaseName + "." + this.tableName.replace(',', '|');
  }
}
 
Example #6
Source File: Name.java    From examples with Apache License 2.0 5 votes vote down vote up
/** {@inheritDoc} */
@Override
public void initialise(FindOptions options) {
  String argPattern = getArguments().get(0);
  if(!caseSensitive) {
    argPattern = argPattern.toLowerCase();
  }
  globPattern = new GlobPattern(argPattern);
}
 
Example #7
Source File: GoogleHadoopFileSystemBase.java    From hadoop-connectors with Apache License 2.0 5 votes vote down vote up
/**
 * Determines based on suitability of {@code fixedPath} whether to use flat globbing logic where
 * we use a single large listing during globStatus to then perform the core globbing logic
 * in-memory.
 */
@VisibleForTesting
boolean couldUseFlatGlob(Path fixedPath) {
  // Only works for filesystems where the base Hadoop Path scheme matches the underlying URI
  // scheme for GCS.
  if (!getUri().getScheme().equals(GoogleCloudStorageFileSystem.SCHEME)) {
    logger.atFinest().log(
        "Flat glob is on, but doesn't work for scheme '%s', using default behavior.",
        getUri().getScheme());
    return false;
  }

  // The full pattern should have a wildcard, otherwise there's no point doing the flat glob.
  GlobPattern fullPattern = new GlobPattern(fixedPath.toString());
  if (!fullPattern.hasWildcard()) {
    logger.atFinest().log(
        "Flat glob is on, but Path '%s' has no wildcard, using default behavior.", fixedPath);
    return false;
  }

  // To use a flat glob, there must be an authority defined.
  if (isNullOrEmpty(fixedPath.toUri().getAuthority())) {
    logger.atFinest().log(
        "Flat glob is on, but Path '%s' has a empty authority, using default behavior.",
        fixedPath);
    return false;
  }

  // And the authority must not contain a wildcard.
  GlobPattern authorityPattern = new GlobPattern(fixedPath.toUri().getAuthority());
  if (authorityPattern.hasWildcard()) {
    logger.atFinest().log(
        "Flat glob is on, but Path '%s' has a wildcard authority, using default behavior.",
        fixedPath);
    return false;
  }

  return true;
}
 
Example #8
Source File: GlobFilter.java    From hadoop with Apache License 2.0 4 votes vote down vote up
@Override
protected Pattern compile(String s) {
  return GlobPattern.compile(s);
}
 
Example #9
Source File: GlobFilter.java    From big-c with Apache License 2.0 4 votes vote down vote up
@Override
protected Pattern compile(String s) {
  return GlobPattern.compile(s);
}
 
Example #10
Source File: PathGlobPattern.java    From parquet-mr with Apache License 2.0 2 votes vote down vote up
/**
 * Compile glob pattern string
 *
 * @param globPattern the glob pattern
 * @return the pattern object
 */
public static Pattern compile(String globPattern) {
  return new GlobPattern(globPattern).compiled();
}