Java Code Examples for org.apache.hadoop.conf.Configuration#getClassLoader()

The following examples show how to use org.apache.hadoop.conf.Configuration#getClassLoader() . These examples are extracted from open source projects. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may want to check out the right sidebar which shows the related API usage.
Example 1
Source Project: hadoop   File: MRAppMaster.java    License: Apache License 2.0 6 votes vote down vote up
/**
 * Executes the given action with the job classloader set as the configuration
 * classloader as well as the thread context class loader if the job
 * classloader is enabled. After the call, the original classloader is
 * restored.
 *
 * If the job classloader is enabled and the code needs to load user-supplied
 * classes via configuration or thread context classloader, this method should
 * be used in order to load them.
 *
 * @param conf the configuration on which the classloader will be set
 * @param action the callable action to be executed
 */
<T> T callWithJobClassLoader(Configuration conf, Action<T> action) {
  // if the job classloader is enabled, we may need it to load the (custom)
  // classes; we make the job classloader available and unset it once it is
  // done
  ClassLoader currentClassLoader = conf.getClassLoader();
  boolean setJobClassLoader =
      jobClassLoader != null && currentClassLoader != jobClassLoader;
  if (setJobClassLoader) {
    MRApps.setClassLoader(jobClassLoader, conf);
  }
  try {
    return action.call(conf);
  } finally {
    if (setJobClassLoader) {
      // restore the original classloader
      MRApps.setClassLoader(currentClassLoader, conf);
    }
  }
}
 
Example 2
Source Project: big-c   File: MRAppMaster.java    License: Apache License 2.0 6 votes vote down vote up
/**
 * Executes the given action with the job classloader set as the configuration
 * classloader as well as the thread context class loader if the job
 * classloader is enabled. After the call, the original classloader is
 * restored.
 *
 * If the job classloader is enabled and the code needs to load user-supplied
 * classes via configuration or thread context classloader, this method should
 * be used in order to load them.
 *
 * @param conf the configuration on which the classloader will be set
 * @param action the callable action to be executed
 */
<T> T callWithJobClassLoader(Configuration conf, Action<T> action) {
  // if the job classloader is enabled, we may need it to load the (custom)
  // classes; we make the job classloader available and unset it once it is
  // done
  ClassLoader currentClassLoader = conf.getClassLoader();
  boolean setJobClassLoader =
      jobClassLoader != null && currentClassLoader != jobClassLoader;
  if (setJobClassLoader) {
    MRApps.setClassLoader(jobClassLoader, conf);
  }
  try {
    return action.call(conf);
  } finally {
    if (setJobClassLoader) {
      // restore the original classloader
      MRApps.setClassLoader(currentClassLoader, conf);
    }
  }
}
 
Example 3
/**
 * Set class loader for current thread and for Confifguration based on 
 * Hadoop home.
 * 
 * @param hdConfDir Hadoop home directory
 * @param conf Hadoop configuration
 * @throws MalformedURLException
 */
private static void setClassLoader(File hdConfDir, Configuration conf) 
throws Exception {
    ClassLoader parent = conf.getClassLoader();
    URL url = hdConfDir.toURI().toURL();
    URL[] urls = new URL[1];
    urls[0] = url;
    ClassLoader classLoader = new URLClassLoader(urls, parent);
    Thread.currentThread().setContextClassLoader(classLoader);
    conf.setClassLoader(classLoader);
}
 
Example 4
Source Project: stratosphere   File: Utils.java    License: Apache License 2.0 5 votes vote down vote up
private static void addPathToConfig(Configuration conf, File path) {
	// chain-in a new classloader
	URL fileUrl = null;
	try {
		fileUrl = path.toURL();
	} catch (MalformedURLException e) {
		throw new RuntimeException("Erroneous config file path", e);
	}
	URL[] urls = {fileUrl};
	ClassLoader cl = new URLClassLoader(urls, conf.getClassLoader());
	conf.setClassLoader(cl);
}
 
Example 5
public static InputStream open(String resource, Configuration conf) {
    ClassLoader loader = conf.getClassLoader();

    if (loader == null) {
        loader = Thread.currentThread().getContextClassLoader();
    }

    if (loader == null) {
        loader = HadoopIOUtils.class.getClassLoader();
    }

    boolean trace = log.isTraceEnabled();

    try {
        // no prefix means classpath
        if (!resource.contains(":")) {

            InputStream result = loader.getResourceAsStream(resource);
            if (result != null) {
                if (trace) {
                    log.trace(String.format("Loaded resource %s from classpath", resource));
                }
                return result;
            }
            // fall back to the distributed cache
            URI[] uris = DistributedCache.getCacheFiles(conf);
            if (uris != null) {
                for (URI uri : uris) {
                    if (uri.toString().contains(resource)) {
                        if (trace) {
                            log.trace(String.format("Loaded resource %s from distributed cache", resource));
                        }
                        return uri.toURL().openStream();
                    }
                }
            }
        }

        // fall back to file system
        Path p = new Path(resource);
        FileSystem fs = p.getFileSystem(conf);
        return fs.open(p);
    } catch (IOException ex) {
        throw new EsHadoopIllegalArgumentException(String.format("Cannot open stream for resource %s", resource));
    }
}