Java Code Examples for org.apache.storm.hdfs.common.rotation.RotationAction

The following examples show how to use org.apache.storm.hdfs.common.rotation.RotationAction. These examples are extracted from open source projects. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source Project: metron   Source File: SourceHandler.java    License: Apache License 2.0 6 votes vote down vote up
protected void rotateOutputFile() throws IOException {
  LOG.debug("Rotating output file...");
  long start = System.currentTimeMillis();
  synchronized (this.writeLock) {
    closeOutputFile();
    // Want to use the callback to make sure we have an accurate count of open files.
    cleanupCallback();

    LOG.debug("Performing {} file rotation actions.", this.rotationActions.size());
    for (RotationAction action : this.rotationActions) {
      action.execute(this.fs, this.currentFile);
    }
  }
  long time = System.currentTimeMillis() - start;
  LOG.info("File rotation took {} ms", time);
}
 
Example 2
Source Project: storm-hdfs   Source File: HdfsState.java    License: Apache License 2.0 6 votes vote down vote up
protected void rotateOutputFile() throws IOException {
    LOG.info("Rotating output file...");
    long start = System.currentTimeMillis();
    synchronized (this.writeLock) {
        closeOutputFile();
        this.rotation++;

        Path newFile = createOutputFile();
        LOG.info("Performing {} file rotation actions.", this.rotationActions.size());
        for (RotationAction action : this.rotationActions) {
            action.execute(this.fs, this.currentFile);
        }
        this.currentFile = newFile;
    }
    long time = System.currentTimeMillis() - start;
    LOG.info("File rotation took {} ms.", time);


}
 
Example 3
Source Project: storm-hdfs   Source File: AbstractHdfsBolt.java    License: Apache License 2.0 6 votes vote down vote up
protected void rotateOutputFile() throws IOException {
    LOG.info("Rotating output file...");
    long start = System.currentTimeMillis();
    synchronized (this.writeLock) {
        closeOutputFile();
        this.rotation++;

        Path newFile = createOutputFile();
        LOG.info("Performing {} file rotation actions.", this.rotationActions.size());
        for (RotationAction action : this.rotationActions) {
            action.execute(this.fs, this.currentFile);
        }
        this.currentFile = newFile;
    }
    long time = System.currentTimeMillis() - start;
    LOG.info("File rotation took {} ms.", time);
}
 
Example 4
Source Project: metron   Source File: SourceHandler.java    License: Apache License 2.0 5 votes vote down vote up
public SourceHandler(List<RotationAction> rotationActions
                    , FileRotationPolicy rotationPolicy
                    , SyncPolicy syncPolicy
                    , FileNameFormat fileNameFormat
                    , SourceHandlerCallback cleanupCallback) throws IOException {
  this.rotationActions = rotationActions;
  this.rotationPolicy = rotationPolicy;
  this.syncPolicy = syncPolicy;
  this.fileNameFormat = fileNameFormat;
  this.cleanupCallback = cleanupCallback;
  initialize();
}
 
Example 5
Source Project: metron   Source File: HdfsWriter.java    License: Apache License 2.0 4 votes vote down vote up
public HdfsWriter addRotationAction(RotationAction action){
  this.rotationActions.add(action);
  return this;
}
 
Example 6
Source Project: storm-hdfs   Source File: HdfsState.java    License: Apache License 2.0 4 votes vote down vote up
public HdfsFileOptions addRotationAction(RotationAction action){
    this.rotationActions.add(action);
    return this;
}
 
Example 7
Source Project: storm-hdfs   Source File: HdfsState.java    License: Apache License 2.0 4 votes vote down vote up
public SequenceFileOptions addRotationAction(RotationAction action){
    this.rotationActions.add(action);
    return this;
}
 
Example 8
Source Project: storm-hdfs   Source File: HdfsBolt.java    License: Apache License 2.0 4 votes vote down vote up
public HdfsBolt addRotationAction(RotationAction action){
    this.rotationActions.add(action);
    return this;
}
 
Example 9
Source Project: storm-hdfs   Source File: SequenceFileBolt.java    License: Apache License 2.0 4 votes vote down vote up
public SequenceFileBolt addRotationAction(RotationAction action){
    this.rotationActions.add(action);
    return this;
}
 
Example 10
Source Project: storm-crawler   Source File: GzipHdfsBolt.java    License: Apache License 2.0 4 votes vote down vote up
public GzipHdfsBolt addRotationAction(RotationAction action) {
    this.rotationActions.add(action);
    return this;
}