Java Code Examples for org.apache.hadoop.conf.Configuration#getLongBytes()

The following examples show how to use org.apache.hadoop.conf.Configuration#getLongBytes() . These examples are extracted from open source projects. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may want to check out the right sidebar which shows the related API usage.
Example 1
Source Project: hadoop   File: DataXceiverServer.java    License: Apache License 2.0 6 votes vote down vote up
DataXceiverServer(PeerServer peerServer, Configuration conf,
    DataNode datanode) {
  
  this.peerServer = peerServer;
  this.datanode = datanode;
  
  this.maxXceiverCount = 
    conf.getInt(DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_KEY,
                DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_DEFAULT);
  
  this.estimateBlockSize = conf.getLongBytes(DFSConfigKeys.DFS_BLOCK_SIZE_KEY,
      DFSConfigKeys.DFS_BLOCK_SIZE_DEFAULT);
  
  //set up parameter for cluster balancing
  this.balanceThrottler = new BlockBalanceThrottler(
      conf.getLong(DFSConfigKeys.DFS_DATANODE_BALANCE_BANDWIDTHPERSEC_KEY,
          DFSConfigKeys.DFS_DATANODE_BALANCE_BANDWIDTHPERSEC_DEFAULT),
      conf.getInt(DFSConfigKeys.DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_KEY,
          DFSConfigKeys.DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_DEFAULT));
}
 
Example 2
Source Project: big-c   File: DataXceiverServer.java    License: Apache License 2.0 6 votes vote down vote up
DataXceiverServer(PeerServer peerServer, Configuration conf,
    DataNode datanode) {
  
  this.peerServer = peerServer;
  this.datanode = datanode;
  
  this.maxXceiverCount = 
    conf.getInt(DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_KEY,
                DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_DEFAULT);
  
  this.estimateBlockSize = conf.getLongBytes(DFSConfigKeys.DFS_BLOCK_SIZE_KEY,
      DFSConfigKeys.DFS_BLOCK_SIZE_DEFAULT);
  
  //set up parameter for cluster balancing
  this.balanceThrottler = new BlockBalanceThrottler(
      conf.getLong(DFSConfigKeys.DFS_DATANODE_BALANCE_BANDWIDTHPERSEC_KEY,
          DFSConfigKeys.DFS_DATANODE_BALANCE_BANDWIDTHPERSEC_DEFAULT),
      conf.getInt(DFSConfigKeys.DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_KEY,
          DFSConfigKeys.DFS_DATANODE_BALANCE_MAX_NUM_CONCURRENT_MOVES_DEFAULT));
}
 
Example 3
Source Project: hadoop   File: BlockSizeParam.java    License: Apache License 2.0 4 votes vote down vote up
/** @return the value or, if it is null, return the default from conf. */
public long getValue(final Configuration conf) {
  return getValue() != null? getValue()
      : conf.getLongBytes(DFS_BLOCK_SIZE_KEY, DFS_BLOCK_SIZE_DEFAULT);
}
 
Example 4
Source Project: big-c   File: BlockSizeParam.java    License: Apache License 2.0 4 votes vote down vote up
/** @return the value or, if it is null, return the default from conf. */
public long getValue(final Configuration conf) {
  return getValue() != null? getValue()
      : conf.getLongBytes(DFS_BLOCK_SIZE_KEY, DFS_BLOCK_SIZE_DEFAULT);
}
 
Example 5
Source Project: stocator   File: COSUtils.java    License: Apache License 2.0 3 votes vote down vote up
/**
 * Get a size property from the configuration: this property must be at least
 * equal to {@link COSConstants#MULTIPART_MIN_SIZE}. If it is too small, it is
 * rounded up to that minimum, and a warning printed.
 *
 * @param conf configuration
 * @param property property name
 * @param defVal default value
 * @return the value, guaranteed to be above the minimum size
 */
public static long getMultipartSizeProperty(Configuration conf,
    String property, long defVal) {
  long partSize = conf.getLongBytes(property, defVal);
  if (partSize < MULTIPART_MIN_SIZE) {
    LOG.warn("{} must be at least 5 MB; configured value is {}", property, partSize);
    partSize = MULTIPART_MIN_SIZE;
  }
  return partSize;
}