Java Code Examples for org.apache.hadoop.conf.Configuration#getLongBytes()

The following examples show how to use org.apache.hadoop.conf.Configuration#getLongBytes() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.

Example 1

Source File: DataXceiverServer.java From hadoop with Apache License 2.0

6 votes

DataXceiverServer(PeerServer peerServer, Configuration conf,
    DataNode datanode) {
  
  this.peerServer = peerServer;
  this.datanode = datanode;
  
  this.maxXceiverCount = 
    conf.getInt(DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_KEY,
                DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_DEFAULT);
  
  this.estimateBlockSize = conf.getLongBytes(DFSConfigKeys.DFS_BLOCK_SIZE_KEY,
      DFSConfigKeys.DFS_BLOCK_SIZE_DEFAULT);
  
  //set up parameter for cluster balancing
  this.balanceThrottler = new BlockBalanceThrottler(
      conf.getLong(DFSConfigKeys.DFS_DATANODE_BALANCE_BANDWIDTHPERSEC_KEY,
          DFSConfigKeys.DFS_DATANODE_BALANCE_BANDWIDTHPERSEC_DEFAULT),
      conf.getInt(DFSConfigKeys.DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_KEY,
          DFSConfigKeys.DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_DEFAULT));
}

Example 2

Source File: DataXceiverServer.java From big-c with Apache License 2.0

6 votes

DataXceiverServer(PeerServer peerServer, Configuration conf,
    DataNode datanode) {
  
  this.peerServer = peerServer;
  this.datanode = datanode;
  
  this.maxXceiverCount = 
    conf.getInt(DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_KEY,
                DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_DEFAULT);
  
  this.estimateBlockSize = conf.getLongBytes(DFSConfigKeys.DFS_BLOCK_SIZE_KEY,
      DFSConfigKeys.DFS_BLOCK_SIZE_DEFAULT);
  
  //set up parameter for cluster balancing
  this.balanceThrottler = new BlockBalanceThrottler(
      conf.getLong(DFSConfigKeys.DFS_DATANODE_BALANCE_BANDWIDTHPERSEC_KEY,
          DFSConfigKeys.DFS_DATANODE_BALANCE_BANDWIDTHPERSEC_DEFAULT),
      conf.getInt(DFSConfigKeys.DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_KEY,
          DFSConfigKeys.DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_DEFAULT));
}

Example 3

Source File: BlockSizeParam.java From hadoop with Apache License 2.0

4 votes

/** @return the value or, if it is null, return the default from conf. */
public long getValue(final Configuration conf) {
  return getValue() != null? getValue()
      : conf.getLongBytes(DFS_BLOCK_SIZE_KEY, DFS_BLOCK_SIZE_DEFAULT);
}

Example 4

Source File: BlockSizeParam.java From big-c with Apache License 2.0

4 votes

/** @return the value or, if it is null, return the default from conf. */
public long getValue(final Configuration conf) {
  return getValue() != null? getValue()
      : conf.getLongBytes(DFS_BLOCK_SIZE_KEY, DFS_BLOCK_SIZE_DEFAULT);
}

Example 5

Source File: COSUtils.java From stocator with Apache License 2.0

3 votes

/**
 * Get a size property from the configuration: this property must be at least
 * equal to {@link COSConstants#MULTIPART_MIN_SIZE}. If it is too small, it is
 * rounded up to that minimum, and a warning printed.
 *
 * @param conf configuration
 * @param property property name
 * @param defVal default value
 * @return the value, guaranteed to be above the minimum size
 */
public static long getMultipartSizeProperty(Configuration conf,
    String property, long defVal) {
  long partSize = conf.getLongBytes(property, defVal);
  if (partSize < MULTIPART_MIN_SIZE) {
    LOG.warn("{} must be at least 5 MB; configured value is {}", property, partSize);
    partSize = MULTIPART_MIN_SIZE;
  }
  return partSize;
}