Java Code Examples for org.apache.hadoop.mapreduce.TaskType#JOB_SETUP

The following examples show how to use org.apache.hadoop.mapreduce.TaskType#JOB_SETUP . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: HadoopV2TaskContext.java    From ignite with Apache License 2.0 6 votes vote down vote up
/**
 * @param type Task type.
 * @return Hadoop task type.
 */
private TaskType taskType(HadoopTaskType type) {
    switch (type) {
        case SETUP:
            return TaskType.JOB_SETUP;
        case MAP:
        case COMBINE:
            return TaskType.MAP;

        case REDUCE:
            return TaskType.REDUCE;

        case COMMIT:
        case ABORT:
            return TaskType.JOB_CLEANUP;

        default:
            return null;
    }
}
 
Example 2
Source File: Version20LogInterfaceUtils.java    From hadoop with Apache License 2.0 5 votes vote down vote up
static TaskType get20TaskType(String taskType) {
  try {
    return TaskType.valueOf(taskType);
  } catch (IllegalArgumentException e) {
    if ("CLEANUP".equals(taskType)) {
      return TaskType.JOB_CLEANUP;
    }

    if ("SETUP".equals(taskType)) {
      return TaskType.JOB_SETUP;
    }

    return null;
  }
}
 
Example 3
Source File: Version20LogInterfaceUtils.java    From big-c with Apache License 2.0 5 votes vote down vote up
static TaskType get20TaskType(String taskType) {
  try {
    return TaskType.valueOf(taskType);
  } catch (IllegalArgumentException e) {
    if ("CLEANUP".equals(taskType)) {
      return TaskType.JOB_CLEANUP;
    }

    if ("SETUP".equals(taskType)) {
      return TaskType.JOB_SETUP;
    }

    return null;
  }
}
 
Example 4
Source File: TaskInProgress.java    From RDFS with Apache License 2.0 5 votes vote down vote up
/**
 * Returns the type of the {@link TaskAttemptID} passed. 
 * The type of an attempt is determined by the nature of the task and not its 
 * id. 
 * For example,
 * - Attempt 'attempt_123_01_m_01_0' might be a job-setup task even though it 
 *   has a _m_ in its id. Hence the task type of this attempt is JOB_SETUP 
 *   instead of MAP.
 * - Similarly reduce attempt 'attempt_123_01_r_01_0' might have failed and is
 *   now supposed to do the task-level cleanup. In such a case this attempt 
 *   will be of type TASK_CLEANUP instead of REDUCE.
 */
TaskType getAttemptType (TaskAttemptID id) {
  if (isCleanupAttempt(id)) {
    return TaskType.TASK_CLEANUP;
  } else if (isJobSetupTask()) {
    return TaskType.JOB_SETUP;
  } else if (isJobCleanupTask()) {
    return TaskType.JOB_CLEANUP;
  } else if (isMapTask()) {
    return TaskType.MAP;
  } else {
    return TaskType.REDUCE;
  }
}
 
Example 5
Source File: TestJobRetire.java    From RDFS with Apache License 2.0 5 votes vote down vote up
private TaskInProgress createAndAddTIP(JobTracker jobtracker, 
                                       JobInProgress jip, TaskType type) {
  JobConf conf = jip.getJobConf();
  JobID id = jip.getJobID();
  // now create a fake tip for this fake job
  TaskInProgress tip = null;
  if (type == TaskType.MAP) {
    tip = new TaskInProgress(id, "dummy", new JobClient.RawSplit(), 
                             conf, jip, 0, 1);
    jip.maps = new TaskInProgress[] {tip};
  } else if (type == TaskType.REDUCE) {
    tip = new TaskInProgress(id, "dummy", jip.desiredMaps(), 0, 
                             conf, jip, 1);
    jip.reduces = new TaskInProgress[] {tip};
  } else if (type == TaskType.JOB_SETUP) {
    tip = 
      new TaskInProgress(id, "dummy", new JobClient.RawSplit(), 
                         conf, jip, 0, 1);
    jip.setup = new TaskInProgress[] {tip};
  } else if (type == TaskType.JOB_CLEANUP) {
    tip = 
      new TaskInProgress(id, "dummy", new JobClient.RawSplit(), 
                         conf, jip, 0, 1);
    jip.cleanup = new TaskInProgress[] {tip};
  }
  return tip;
}
 
Example 6
Source File: HadoopFormats.java    From beam with Apache License 2.0 2 votes vote down vote up
/**
 * Creates new setup {@link TaskAttemptContext} from hadoop {@link Configuration} and {@link
 * JobID}.
 *
 * @param conf hadoop {@link Configuration}
 * @param jobID jobId of the created {@link TaskAttemptContext}
 * @return new setup {@link TaskAttemptContext}
 */
static TaskAttemptContext createSetupTaskContext(Configuration conf, JobID jobID) {
  final TaskID taskId = new TaskID(jobID, TaskType.JOB_SETUP, 0);
  return createTaskAttemptContext(conf, new TaskAttemptID(taskId, 0));
}