org.apache.hadoop.classification.InterfaceAudience Java Examples

The following examples show how to use org.apache.hadoop.classification.InterfaceAudience. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: HttpServer.java    From hadoop with Apache License 2.0 6 votes vote down vote up
@InterfaceAudience.Private
public static Connector createDefaultChannelConnector() {
  SelectChannelConnector ret = new SelectChannelConnectorWithSafeStartup();
  ret.setLowResourceMaxIdleTime(10000);
  ret.setAcceptQueueSize(128);
  ret.setResolveNames(false);
  ret.setUseDirectBuffers(false);
  if(Shell.WINDOWS) {
    // result of setting the SO_REUSEADDR flag is different on Windows
    // http://msdn.microsoft.com/en-us/library/ms740621(v=vs.85).aspx
    // without this 2 NN's can start on the same machine and listen on 
    // the same port with indeterminate routing of incoming requests to them
    ret.setReuseAddress(false);
  }
  ret.setHeaderBufferSize(1024*64);
  return ret;
}
 
Example #2
Source File: ServerRMProxy.java    From hadoop with Apache License 2.0 6 votes vote down vote up
@InterfaceAudience.Private
@Override
protected InetSocketAddress getRMAddress(YarnConfiguration conf,
                                         Class<?> protocol) {
  if (protocol == ResourceTracker.class) {
    return conf.getSocketAddr(
      YarnConfiguration.RM_RESOURCE_TRACKER_ADDRESS,
      YarnConfiguration.DEFAULT_RM_RESOURCE_TRACKER_ADDRESS,
      YarnConfiguration.DEFAULT_RM_RESOURCE_TRACKER_PORT);
  } else {
    String message = "Unsupported protocol found when creating the proxy " +
        "connection to ResourceManager: " +
        ((protocol != null) ? protocol.getClass().getName() : "null");
    LOG.error(message);
    throw new IllegalStateException(message);
  }
}
 
Example #3
Source File: ActiveStandbyElector.java    From hadoop with Apache License 2.0 6 votes vote down vote up
@InterfaceAudience.Private
public synchronized void terminateConnection() {
  if (zkClient == null) {
    return;
  }
  LOG.debug("Terminating ZK connection for " + this);
  ZooKeeper tempZk = zkClient;
  zkClient = null;
  watcher = null;
  try {
    tempZk.close();
  } catch(InterruptedException e) {
    LOG.warn(e);
  }
  zkConnectionState = ConnectionState.TERMINATED;
  wantToBeInElection = false;
}
 
Example #4
Source File: SecurityUtil.java    From big-c with Apache License 2.0 6 votes vote down vote up
/**
 * Convert Kerberos principal name pattern to valid Kerberos principal names.
 * This method is similar to {@link #getServerPrincipal(String, String)},
 * except 1) the reverse DNS lookup from addr to hostname is done only when
 * necessary, 2) param addr can't be null (no default behavior of using local
 * hostname when addr is null).
 * 
 * @param principalConfig
 *          Kerberos principal name pattern to convert
 * @param addr
 *          InetAddress of the host used for substitution
 * @return converted Kerberos principal name
 * @throws IOException if the client address cannot be determined
 */
@InterfaceAudience.Public
@InterfaceStability.Evolving
public static String getServerPrincipal(String principalConfig,
    InetAddress addr) throws IOException {
  String[] components = getComponents(principalConfig);
  if (components == null || components.length != 3
      || !components[1].equals(HOSTNAME_PATTERN)) {
    return principalConfig;
  } else {
    if (addr == null) {
      throw new IOException("Can't replace " + HOSTNAME_PATTERN
          + " pattern since client address is null");
    }
    return replacePattern(components, addr.getCanonicalHostName());
  }
}
 
Example #5
Source File: MetricsRegistry.java    From big-c with Apache License 2.0 6 votes vote down vote up
@InterfaceAudience.Private
public synchronized MutableRate newRate(String name, String desc,
    boolean extended, boolean returnExisting) {
  if (returnExisting) {
    MutableMetric rate = metricsMap.get(name);
    if (rate != null) {
      if (rate instanceof MutableRate) return (MutableRate) rate;
      throw new MetricsException("Unexpected metrics type "+ rate.getClass()
                                 +" for "+ name);
    }
  }
  checkMetricName(name);
  MutableRate ret = new MutableRate(name, desc, extended);
  metricsMap.put(name, ret);
  return ret;
}
 
Example #6
Source File: UserGroupInformation.java    From big-c with Apache License 2.0 6 votes vote down vote up
/**
 * Run the given action as the user, potentially throwing an exception.
 * @param <T> the return type of the run method
 * @param action the method to execute
 * @return the value from the run method
 * @throws IOException if the action throws an IOException
 * @throws Error if the action throws an Error
 * @throws RuntimeException if the action throws a RuntimeException
 * @throws InterruptedException if the action throws an InterruptedException
 * @throws UndeclaredThrowableException if the action throws something else
 */
@InterfaceAudience.Public
@InterfaceStability.Evolving
public <T> T doAs(PrivilegedExceptionAction<T> action
                  ) throws IOException, InterruptedException {
  try {
    logPrivilegedAction(subject, action);
    return Subject.doAs(subject, action);
  } catch (PrivilegedActionException pae) {
    Throwable cause = pae.getCause();
    if (LOG.isDebugEnabled()) {
      LOG.debug("PrivilegedActionException as:" + this + " cause:" + cause);
    }
    if (cause instanceof IOException) {
      throw (IOException) cause;
    } else if (cause instanceof Error) {
      throw (Error) cause;
    } else if (cause instanceof RuntimeException) {
      throw (RuntimeException) cause;
    } else if (cause instanceof InterruptedException) {
      throw (InterruptedException) cause;
    } else {
      throw new UndeclaredThrowableException(cause);
    }
  }
}
 
Example #7
Source File: DataNode.java    From big-c with Apache License 2.0 5 votes vote down vote up
/**
 * Creates a dummy DataNode for testing purpose.
 */
@VisibleForTesting
@InterfaceAudience.LimitedPrivate("HDFS")
DataNode(final Configuration conf) {
  super(conf);
  this.blockScanner = new BlockScanner(this, conf);
  this.fileDescriptorPassingDisabledReason = null;
  this.maxNumberOfBlocksToLog = 0;
  this.confVersion = null;
  this.usersWithLocalPathAccess = null;
  this.connectToDnViaHostname = false;
  this.getHdfsBlockLocationsEnabled = false;
  this.pipelineSupportECN = false;
}
 
Example #8
Source File: HAUtil.java    From big-c with Apache License 2.0 5 votes vote down vote up
@InterfaceAudience.Private
@VisibleForTesting
static String getConfKeyForRMInstance(String prefix, Configuration conf) {
  if (!YarnConfiguration.getServiceAddressConfKeys(conf).contains(prefix)) {
    return prefix;
  } else {
    String RMId = getRMHAId(conf);
    checkAndSetRMRPCAddress(prefix, RMId, conf);
    return addSuffix(prefix, RMId);
  }
}
 
Example #9
Source File: UserGroupInformation.java    From hadoop with Apache License 2.0 5 votes vote down vote up
/**
 * Return the current user, including any doAs in the current stack.
 * @return the current user
 * @throws IOException if login fails
 */
@InterfaceAudience.Public
@InterfaceStability.Evolving
public synchronized
static UserGroupInformation getCurrentUser() throws IOException {
  AccessControlContext context = AccessController.getContext();
  Subject subject = Subject.getSubject(context);
  if (subject == null || subject.getPrincipals(User.class).isEmpty()) {
    return getLoginUser();
  } else {
    return new UserGroupInformation(subject);
  }
}
 
Example #10
Source File: AvroReflectSerialization.java    From hadoop with Apache License 2.0 5 votes vote down vote up
@InterfaceAudience.Private
@Override
public DatumReader getReader(Class<Object> clazz) {
  try {
    return new ReflectDatumReader(clazz);
  } catch (Exception e) {
    throw new RuntimeException(e);
  }
}
 
Example #11
Source File: SaslRpcServer.java    From hadoop with Apache License 2.0 5 votes vote down vote up
@InterfaceAudience.Private
@InterfaceStability.Unstable
public SaslRpcServer(AuthMethod authMethod) throws IOException {
  this.authMethod = authMethod;
  mechanism = authMethod.getMechanismName();    
  switch (authMethod) {
    case SIMPLE: {
      return; // no sasl for simple
    }
    case TOKEN: {
      protocol = "";
      serverId = SaslRpcServer.SASL_DEFAULT_REALM;
      break;
    }
    case KERBEROS: {
      String fullName = UserGroupInformation.getCurrentUser().getUserName();
      if (LOG.isDebugEnabled())
        LOG.debug("Kerberos principal name is " + fullName);
      // don't use KerberosName because we don't want auth_to_local
      String[] parts = fullName.split("[/@]", 3);
      protocol = parts[0];
      // should verify service host is present here rather than in create()
      // but lazy tests are using a UGI that isn't a SPN...
      serverId = (parts.length < 2) ? "" : parts[1];
      break;
    }
    default:
      // we should never be able to get here
      throw new AccessControlException(
          "Server does not support SASL " + authMethod);
  }
}
 
Example #12
Source File: ContentSummary.java    From big-c with Apache License 2.0 5 votes vote down vote up
@Override
@InterfaceAudience.Private
public void write(DataOutput out) throws IOException {
  out.writeLong(length);
  out.writeLong(fileCount);
  out.writeLong(directoryCount);
  out.writeLong(quota);
  out.writeLong(spaceConsumed);
  out.writeLong(spaceQuota);
}
 
Example #13
Source File: AbstractCounters.java    From hadoop with Apache License 2.0 5 votes vote down vote up
/**
 * Find the file system counter for the given scheme and enum.
 * @param scheme of the file system
 * @param key the enum of the counter
 * @return the file system counter
 */
@InterfaceAudience.Private
public synchronized C findCounter(String scheme, FileSystemCounter key) {
  return ((FileSystemCounterGroup<C>) getGroup(
      FileSystemCounter.class.getName()).getUnderlyingGroup()).
      findCounter(scheme, key);
}
 
Example #14
Source File: UserGroupInformation.java    From big-c with Apache License 2.0 5 votes vote down vote up
@InterfaceAudience.Private
@InterfaceStability.Unstable
@VisibleForTesting
public synchronized static void setLoginUser(UserGroupInformation ugi) {
  // if this is to become stable, should probably logout the currently
  // logged in ugi if it's different
  loginUser = ugi;
}
 
Example #15
Source File: UserGroupInformation.java    From big-c with Apache License 2.0 5 votes vote down vote up
@InterfaceAudience.Private
@VisibleForTesting
static void reset() {
  authenticationMethod = null;
  conf = null;
  groups = null;
  setLoginUser(null);
  HadoopKerberosName.setRules(null);
}
 
Example #16
Source File: SaslRpcServer.java    From big-c with Apache License 2.0 5 votes vote down vote up
@InterfaceAudience.Private
@InterfaceStability.Unstable
public SaslRpcServer(AuthMethod authMethod) throws IOException {
  this.authMethod = authMethod;
  mechanism = authMethod.getMechanismName();    
  switch (authMethod) {
    case SIMPLE: {
      return; // no sasl for simple
    }
    case TOKEN: {
      protocol = "";
      serverId = SaslRpcServer.SASL_DEFAULT_REALM;
      break;
    }
    case KERBEROS: {
      String fullName = UserGroupInformation.getCurrentUser().getUserName();
      if (LOG.isDebugEnabled())
        LOG.debug("Kerberos principal name is " + fullName);
      // don't use KerberosName because we don't want auth_to_local
      String[] parts = fullName.split("[/@]", 3);
      protocol = parts[0];
      // should verify service host is present here rather than in create()
      // but lazy tests are using a UGI that isn't a SPN...
      serverId = (parts.length < 2) ? "" : parts[1];
      break;
    }
    default:
      // we should never be able to get here
      throw new AccessControlException(
          "Server does not support SASL " + authMethod);
  }
}
 
Example #17
Source File: SecurityUtil.java    From hadoop with Apache License 2.0 5 votes vote down vote up
/**
 * For use only by tests and initialization
 */
@InterfaceAudience.Private
@VisibleForTesting
public static void setTokenServiceUseIp(boolean flag) {
  useIpForTokenService = flag;
  hostResolver = !useIpForTokenService
      ? new QualifiedHostResolver()
      : new StandardHostResolver();
}
 
Example #18
Source File: UserGroupInformation.java    From big-c with Apache License 2.0 5 votes vote down vote up
/**
 * Create a user from a login name. It is intended to be used for remote
 * users in RPC, since it won't have any credentials.
 * @param user the full user principal name, must not be empty or null
 * @return the UserGroupInformation for the remote user.
 */
@InterfaceAudience.Public
@InterfaceStability.Evolving
public static UserGroupInformation createRemoteUser(String user, AuthMethod authMethod) {
  if (user == null || user.isEmpty()) {
    throw new IllegalArgumentException("Null user");
  }
  Subject subject = new Subject();
  subject.getPrincipals().add(new User(user));
  UserGroupInformation result = new UserGroupInformation(subject);
  result.setAuthenticationMethod(authMethod);
  return result;
}
 
Example #19
Source File: HAUtil.java    From hadoop with Apache License 2.0 5 votes vote down vote up
@InterfaceAudience.Private
@VisibleForTesting
static String getConfKeyForRMInstance(String prefix, Configuration conf) {
  if (!YarnConfiguration.getServiceAddressConfKeys(conf).contains(prefix)) {
    return prefix;
  } else {
    String RMId = getRMHAId(conf);
    checkAndSetRMRPCAddress(prefix, RMId, conf);
    return addSuffix(prefix, RMId);
  }
}
 
Example #20
Source File: LocalDirAllocator.java    From hadoop with Apache License 2.0 5 votes vote down vote up
/**
 * Removes the context from the context config items
 * 
 * @param contextCfgItemName
 */
@Deprecated
@InterfaceAudience.LimitedPrivate({"MapReduce"})
public static void removeContext(String contextCfgItemName) {
  synchronized (contexts) {
    contexts.remove(contextCfgItemName);
  }
}
 
Example #21
Source File: UserGroupInformation.java    From big-c with Apache License 2.0 5 votes vote down vote up
/**
 * Return the current user, including any doAs in the current stack.
 * @return the current user
 * @throws IOException if login fails
 */
@InterfaceAudience.Public
@InterfaceStability.Evolving
public synchronized
static UserGroupInformation getCurrentUser() throws IOException {
  AccessControlContext context = AccessController.getContext();
  Subject subject = Subject.getSubject(context);
  if (subject == null || subject.getPrincipals(User.class).isEmpty()) {
    return getLoginUser();
  } else {
    return new UserGroupInformation(subject);
  }
}
 
Example #22
Source File: CounterStatsDClient.java    From datawave with Apache License 2.0 4 votes vote down vote up
@Override
@InterfaceAudience.Private
public Counter getUnderlyingCounter() {
    return delegate;
}
 
Example #23
Source File: JavaSerialization.java    From big-c with Apache License 2.0 4 votes vote down vote up
@Override
@InterfaceAudience.Private
public Serializer<Serializable> getSerializer(Class<Serializable> c) {
  return new JavaSerializationSerializer();
}
 
Example #24
Source File: SaslRpcClient.java    From hadoop with Apache License 2.0 4 votes vote down vote up
@InterfaceAudience.Private
public AuthMethod getAuthMethod() {
  return authMethod;
}
 
Example #25
Source File: Server.java    From big-c with Apache License 2.0 4 votes vote down vote up
@InterfaceAudience.LimitedPrivate({"HDFS", "MapReduce"})
public ServiceAuthorizationManager getServiceAuthorizationManager() {
  return serviceAuthorizationManager;
}
 
Example #26
Source File: SaslRpcClient.java    From big-c with Apache License 2.0 4 votes vote down vote up
@VisibleForTesting
@InterfaceAudience.Private
public Object getNegotiatedProperty(String key) {
  return (saslClient != null) ? saslClient.getNegotiatedProperty(key) : null;
}
 
Example #27
Source File: JobQueueInfo.java    From big-c with Apache License 2.0 4 votes vote down vote up
@InterfaceAudience.Private
public void setProperties(Properties props) {
  super.setProperties(props);
}
 
Example #28
Source File: WritableRpcEngine.java    From hadoop with Apache License 2.0 4 votes vote down vote up
@InterfaceAudience.Private
@InterfaceStability.Unstable
static Client getClient(Configuration conf) {
  return CLIENTS.getClient(conf);
}
 
Example #29
Source File: Count.java    From hadoop with Apache License 2.0 4 votes vote down vote up
/**
 * Should sizes be shown in human readable format rather than bytes?
 * @return true if human readable format
 */
@InterfaceAudience.Private
boolean isHumanReadable() {
  return humanReadable;
}
 
Example #30
Source File: AvroReflectSerialization.java    From big-c with Apache License 2.0 4 votes vote down vote up
@InterfaceAudience.Private
@Override
public Schema getSchema(Object t) {
  return ReflectData.get().getSchema(t.getClass());
}