Java Code Examples for org.apache.hadoop.conf.Configuration#getClassLoader()

The following examples show how to use org.apache.hadoop.conf.Configuration#getClassLoader() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: MRAppMaster.java    From hadoop with Apache License 2.0 6 votes vote down vote up 
/**
 * Executes the given action with the job classloader set as the configuration
 * classloader as well as the thread context class loader if the job
 * classloader is enabled. After the call, the original classloader is
 * restored.
 *
 * If the job classloader is enabled and the code needs to load user-supplied
 * classes via configuration or thread context classloader, this method should
 * be used in order to load them.
 *
 * @param conf the configuration on which the classloader will be set
 * @param action the callable action to be executed
 */
<T> T callWithJobClassLoader(Configuration conf, Action<T> action) {
  // if the job classloader is enabled, we may need it to load the (custom)
  // classes; we make the job classloader available and unset it once it is
  // done
  ClassLoader currentClassLoader = conf.getClassLoader();
  boolean setJobClassLoader =
      jobClassLoader != null && currentClassLoader != jobClassLoader;
  if (setJobClassLoader) {
    MRApps.setClassLoader(jobClassLoader, conf);
  }
  try {
    return action.call(conf);
  } finally {
    if (setJobClassLoader) {
      // restore the original classloader
      MRApps.setClassLoader(currentClassLoader, conf);
    }
  }
}
Example 2
Source File: MRAppMaster.java    From big-c with Apache License 2.0 6 votes vote down vote up 
/**
 * Executes the given action with the job classloader set as the configuration
 * classloader as well as the thread context class loader if the job
 * classloader is enabled. After the call, the original classloader is
 * restored.
 *
 * If the job classloader is enabled and the code needs to load user-supplied
 * classes via configuration or thread context classloader, this method should
 * be used in order to load them.
 *
 * @param conf the configuration on which the classloader will be set
 * @param action the callable action to be executed
 */
<T> T callWithJobClassLoader(Configuration conf, Action<T> action) {
  // if the job classloader is enabled, we may need it to load the (custom)
  // classes; we make the job classloader available and unset it once it is
  // done
  ClassLoader currentClassLoader = conf.getClassLoader();
  boolean setJobClassLoader =
      jobClassLoader != null && currentClassLoader != jobClassLoader;
  if (setJobClassLoader) {
    MRApps.setClassLoader(jobClassLoader, conf);
  }
  try {
    return action.call(conf);
  } finally {
    if (setJobClassLoader) {
      // restore the original classloader
      MRApps.setClassLoader(currentClassLoader, conf);
    }
  }
}
Example 3
Source File: ContentPump.java    From marklogic-contentpump with Apache License 2.0 5 votes vote down vote up 
/**
 * Set class loader for current thread and for Confifguration based on 
 * Hadoop home.
 * 
 * @param hdConfDir Hadoop home directory
 * @param conf Hadoop configuration
 * @throws MalformedURLException
 */
private static void setClassLoader(File hdConfDir, Configuration conf) 
throws Exception {
    ClassLoader parent = conf.getClassLoader();
    URL url = hdConfDir.toURI().toURL();
    URL[] urls = new URL[1];
    urls[0] = url;
    ClassLoader classLoader = new URLClassLoader(urls, parent);
    Thread.currentThread().setContextClassLoader(classLoader);
    conf.setClassLoader(classLoader);
}
Example 4
Source File: Utils.java    From stratosphere with Apache License 2.0 5 votes vote down vote up 
private static void addPathToConfig(Configuration conf, File path) {
	// chain-in a new classloader
	URL fileUrl = null;
	try {
		fileUrl = path.toURL();
	} catch (MalformedURLException e) {
		throw new RuntimeException("Erroneous config file path", e);
	}
	URL[] urls = {fileUrl};
	ClassLoader cl = new URLClassLoader(urls, conf.getClassLoader());
	conf.setClassLoader(cl);
}
Example 5
Source File: HadoopIOUtils.java    From elasticsearch-hadoop with Apache License 2.0 4 votes vote down vote up 
public static InputStream open(String resource, Configuration conf) {
    ClassLoader loader = conf.getClassLoader();

    if (loader == null) {
        loader = Thread.currentThread().getContextClassLoader();
    }

    if (loader == null) {
        loader = HadoopIOUtils.class.getClassLoader();
    }

    boolean trace = log.isTraceEnabled();

    try {
        // no prefix means classpath
        if (!resource.contains(":")) {

            InputStream result = loader.getResourceAsStream(resource);
            if (result != null) {
                if (trace) {
                    log.trace(String.format("Loaded resource %s from classpath", resource));
                }
                return result;
            }
            // fall back to the distributed cache
            URI[] uris = DistributedCache.getCacheFiles(conf);
            if (uris != null) {
                for (URI uri : uris) {
                    if (uri.toString().contains(resource)) {
                        if (trace) {
                            log.trace(String.format("Loaded resource %s from distributed cache", resource));
                        }
                        return uri.toURL().openStream();
                    }
                }
            }
        }

        // fall back to file system
        Path p = new Path(resource);
        FileSystem fs = p.getFileSystem(conf);
        return fs.open(p);
    } catch (IOException ex) {
        throw new EsHadoopIllegalArgumentException(String.format("Cannot open stream for resource %s", resource));
    }
}