Java Code Examples for org.apache.hadoop.fs.FileUtil#replaceFile()

The following examples show how to use org.apache.hadoop.fs.FileUtil#replaceFile() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: Journal.java    From hadoop with Apache License 2.0 6 votes vote down vote up 
/**
 * In the case the node crashes in between downloading a log segment
 * and persisting the associated paxos recovery data, the log segment
 * will be left in its temporary location on disk. Given the paxos data,
 * we can check if this was indeed the case, and "roll forward"
 * the atomic operation.
 * 
 * See the inline comments in
 * {@link #acceptRecovery(RequestInfo, SegmentStateProto, URL)} for more
 * details.
 *
 * @throws IOException if the temporary file is unable to be renamed into
 * place
 */
private void completeHalfDoneAcceptRecovery(
    PersistedRecoveryPaxosData paxosData) throws IOException {
  if (paxosData == null) {
    return;
  }

  long segmentId = paxosData.getSegmentState().getStartTxId();
  long epoch = paxosData.getAcceptedInEpoch();
  
  File tmp = storage.getSyncLogTemporaryFile(segmentId, epoch);
  
  if (tmp.exists()) {
    File dst = storage.getInProgressEditLog(segmentId);
    LOG.info("Rolling forward previously half-completed synchronization: " +
        tmp + " -> " + dst);
    FileUtil.replaceFile(tmp, dst);
  }
}
Example 2
Source File: Journal.java    From big-c with Apache License 2.0 6 votes vote down vote up 
/**
 * In the case the node crashes in between downloading a log segment
 * and persisting the associated paxos recovery data, the log segment
 * will be left in its temporary location on disk. Given the paxos data,
 * we can check if this was indeed the case, and "roll forward"
 * the atomic operation.
 * 
 * See the inline comments in
 * {@link #acceptRecovery(RequestInfo, SegmentStateProto, URL)} for more
 * details.
 *
 * @throws IOException if the temporary file is unable to be renamed into
 * place
 */
private void completeHalfDoneAcceptRecovery(
    PersistedRecoveryPaxosData paxosData) throws IOException {
  if (paxosData == null) {
    return;
  }

  long segmentId = paxosData.getSegmentState().getStartTxId();
  long epoch = paxosData.getAcceptedInEpoch();
  
  File tmp = storage.getSyncLogTemporaryFile(segmentId, epoch);
  
  if (tmp.exists()) {
    File dst = storage.getInProgressEditLog(segmentId);
    LOG.info("Rolling forward previously half-completed synchronization: " +
        tmp + " -> " + dst);
    FileUtil.replaceFile(tmp, dst);
  }
}
Example 3
Source File: DatanodeBlockInfo.java    From RDFS with Apache License 2.0 6 votes vote down vote up 
/**
 * Copy specified file into a temporary file. Then rename the
 * temporary file to the original name. This will cause any
 * hardlinks to the original file to be removed. The temporary
 * files are created in the detachDir. The temporary files will
 * be recovered (especially on Windows) on datanode restart.
 */
private void detachFile(int namespaceId, File file, Block b) throws IOException {
  File tmpFile = volume.createDetachFile(namespaceId, b, file.getName());
  try {
    IOUtils.copyBytes(new FileInputStream(file),
                      new FileOutputStream(tmpFile),
                      16*1024, true);
    if (file.length() != tmpFile.length()) {
      throw new IOException("Copy of file " + file + " size " + file.length()+
                            " into file " + tmpFile +
                            " resulted in a size of " + tmpFile.length());
    }
    FileUtil.replaceFile(tmpFile, file);
  } catch (IOException e) {
    boolean done = tmpFile.delete();
    if (!done) {
      DataNode.LOG.info("detachFile failed to delete temporary file " +
                        tmpFile);
    }
    throw e;
  }
}
Example 4
Source File: DatanodeBlockInfo.java    From hadoop-gpu with Apache License 2.0 6 votes vote down vote up 
/**
 * Copy specified file into a temporary file. Then rename the
 * temporary file to the original name. This will cause any
 * hardlinks to the original file to be removed. The temporary
 * files are created in the detachDir. The temporary files will
 * be recovered (especially on Windows) on datanode restart.
 */
private void detachFile(File file, Block b) throws IOException {
  File tmpFile = volume.createDetachFile(b, file.getName());
  try {
    IOUtils.copyBytes(new FileInputStream(file),
                      new FileOutputStream(tmpFile),
                      16*1024, true);
    if (file.length() != tmpFile.length()) {
      throw new IOException("Copy of file " + file + " size " + file.length()+
                            " into file " + tmpFile +
                            " resulted in a size of " + tmpFile.length());
    }
    FileUtil.replaceFile(tmpFile, file);
  } catch (IOException e) {
    boolean done = tmpFile.delete();
    if (!done) {
      DataNode.LOG.info("detachFile failed to delete temporary file " +
                        tmpFile);
    }
    throw e;
  }
}
Example 5
Source File: ReplicaInfo.java    From hadoop with Apache License 2.0 5 votes vote down vote up 
/**
 * Copy specified file into a temporary file. Then rename the
 * temporary file to the original name. This will cause any
 * hardlinks to the original file to be removed. The temporary
 * files are created in the same directory. The temporary files will
 * be recovered (especially on Windows) on datanode restart.
 */
private void unlinkFile(File file, Block b) throws IOException {
  File tmpFile = DatanodeUtil.createTmpFile(b, DatanodeUtil.getUnlinkTmpFile(file));
  try {
    FileInputStream in = new FileInputStream(file);
    try {
      FileOutputStream out = new FileOutputStream(tmpFile);
      try {
        IOUtils.copyBytes(in, out, 16*1024);
      } finally {
        out.close();
      }
    } finally {
      in.close();
    }
    if (file.length() != tmpFile.length()) {
      throw new IOException("Copy of file " + file + " size " + file.length()+
                            " into file " + tmpFile +
                            " resulted in a size of " + tmpFile.length());
    }
    FileUtil.replaceFile(tmpFile, file);
  } catch (IOException e) {
    boolean done = tmpFile.delete();
    if (!done) {
      DataNode.LOG.info("detachFile failed to delete temporary file " +
                        tmpFile);
    }
    throw e;
  }
}
Example 6
Source File: ReplicaInfo.java    From big-c with Apache License 2.0 5 votes vote down vote up 
/**
 * Copy specified file into a temporary file. Then rename the
 * temporary file to the original name. This will cause any
 * hardlinks to the original file to be removed. The temporary
 * files are created in the same directory. The temporary files will
 * be recovered (especially on Windows) on datanode restart.
 */
private void unlinkFile(File file, Block b) throws IOException {
  File tmpFile = DatanodeUtil.createTmpFile(b, DatanodeUtil.getUnlinkTmpFile(file));
  try {
    FileInputStream in = new FileInputStream(file);
    try {
      FileOutputStream out = new FileOutputStream(tmpFile);
      try {
        IOUtils.copyBytes(in, out, 16*1024);
      } finally {
        out.close();
      }
    } finally {
      in.close();
    }
    if (file.length() != tmpFile.length()) {
      throw new IOException("Copy of file " + file + " size " + file.length()+
                            " into file " + tmpFile +
                            " resulted in a size of " + tmpFile.length());
    }
    FileUtil.replaceFile(tmpFile, file);
  } catch (IOException e) {
    boolean done = tmpFile.delete();
    if (!done) {
      DataNode.LOG.info("detachFile failed to delete temporary file " +
                        tmpFile);
    }
    throw e;
  }
}