Java Code Examples for org.apache.hadoop.fs.FileSystem#getAllStatistics()
The following examples show how to use
org.apache.hadoop.fs.FileSystem#getAllStatistics() .
You can vote up the ones you like or vote down the ones you don't like,
and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: Task.java From hadoop with Apache License 2.0 | 5 votes |
/** * Gets a handle to the Statistics instance based on the scheme associated * with path. * * @param path the path. * @param conf the configuration to extract the scheme from if not part of * the path. * @return a Statistics instance, or null if none is found for the scheme. */ protected static List<Statistics> getFsStatistics(Path path, Configuration conf) throws IOException { List<Statistics> matchedStats = new ArrayList<FileSystem.Statistics>(); path = path.getFileSystem(conf).makeQualified(path); String scheme = path.toUri().getScheme(); for (Statistics stats : FileSystem.getAllStatistics()) { if (stats.getScheme().equals(scheme)) { matchedStats.add(stats); } } return matchedStats; }
Example 2
Source File: Task.java From big-c with Apache License 2.0 | 5 votes |
/** * Gets a handle to the Statistics instance based on the scheme associated * with path. * * @param path the path. * @param conf the configuration to extract the scheme from if not part of * the path. * @return a Statistics instance, or null if none is found for the scheme. */ protected static List<Statistics> getFsStatistics(Path path, Configuration conf) throws IOException { List<Statistics> matchedStats = new ArrayList<FileSystem.Statistics>(); path = path.getFileSystem(conf).makeQualified(path); String scheme = path.toUri().getScheme(); for (Statistics stats : FileSystem.getAllStatistics()) { if (stats.getScheme().equals(scheme)) { matchedStats.add(stats); } } return matchedStats; }
Example 3
Source File: Task.java From RDFS with Apache License 2.0 | 5 votes |
private synchronized void updateCounters() { for(Statistics stat: FileSystem.getAllStatistics()) { String uriScheme = stat.getScheme(); FileSystemStatisticUpdater updater = statisticUpdaters.get(uriScheme); if(updater==null) {//new FileSystem has been found in the cache updater = new FileSystemStatisticUpdater(uriScheme, stat); statisticUpdaters.put(uriScheme, updater); } updater.updateCounters(); } updateResourceCounters(); }
Example 4
Source File: MRTask.java From incubator-tez with Apache License 2.0 | 5 votes |
/** * Gets a handle to the Statistics instance based on the scheme associated * with path. * * @param path the path. * @param conf the configuration to extract the scheme from if not part of * the path. * @return a Statistics instance, or null if none is found for the scheme. */ @Private public static List<Statistics> getFsStatistics(Path path, Configuration conf) throws IOException { List<Statistics> matchedStats = new ArrayList<FileSystem.Statistics>(); path = path.getFileSystem(conf).makeQualified(path); String scheme = path.toUri().getScheme(); for (Statistics stats : FileSystem.getAllStatistics()) { if (stats.getScheme().equals(scheme)) { matchedStats.add(stats); } } return matchedStats; }
Example 5
Source File: Utils.java From incubator-tez with Apache License 2.0 | 5 votes |
/** * Gets a handle to the Statistics instance based on the scheme associated * with path. * * @param path the path. * @param conf the configuration to extract the scheme from if not part of * the path. * @return a Statistics instance, or null if none is found for the scheme. */ @Private public static List<Statistics> getFsStatistics(Path path, Configuration conf) throws IOException { List<Statistics> matchedStats = new ArrayList<FileSystem.Statistics>(); path = path.getFileSystem(conf).makeQualified(path); String scheme = path.toUri().getScheme(); for (Statistics stats : FileSystem.getAllStatistics()) { if (stats.getScheme().equals(scheme)) { matchedStats.add(stats); } } return matchedStats; }
Example 6
Source File: Utils.java From tez with Apache License 2.0 | 5 votes |
/** * Gets a handle to the Statistics instance based on the scheme associated * with path. * * @param path the path. * @param conf the configuration to extract the scheme from if not part of * the path. * @return a Statistics instance, or null if none is found for the scheme. */ @Private public static List<Statistics> getFsStatistics(Path path, Configuration conf) throws IOException { List<Statistics> matchedStats = new ArrayList<FileSystem.Statistics>(); path = path.getFileSystem(conf).makeQualified(path); String scheme = path.toUri().getScheme(); for (Statistics stats : FileSystem.getAllStatistics()) { if (stats.getScheme().equals(scheme)) { matchedStats.add(stats); } } return matchedStats; }
Example 7
Source File: Task.java From hadoop-gpu with Apache License 2.0 | 5 votes |
private synchronized void updateCounters() { for(Statistics stat: FileSystem.getAllStatistics()) { String uriScheme = stat.getScheme(); FileSystemStatisticUpdater updater = statisticUpdaters.get(uriScheme); if(updater==null) {//new FileSystem has been found in the cache updater = new FileSystemStatisticUpdater(uriScheme, stat); statisticUpdaters.put(uriScheme, updater); } updater.updateCounters(); } }