Java Code Examples for org.apache.flink.api.common.io.FileOutputFormat#setWriteMode()

The following examples show how to use org.apache.flink.api.common.io.FileOutputFormat#setWriteMode() . These examples are extracted from open source projects. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source Project: Flink-CEPplus   File: TaskTestBase.java    License: Apache License 2.0 5 votes vote down vote up
public void registerFileOutputTask(FileOutputFormat<Record> outputFormat, String outPath) {
	TaskConfig dsConfig = new TaskConfig(this.mockEnv.getTaskConfiguration());

	outputFormat.setOutputFilePath(new Path(outPath));
	outputFormat.setWriteMode(WriteMode.OVERWRITE);

	dsConfig.setStubWrapper(new UserCodeObjectWrapper<>(outputFormat));
}
 
Example 2
Source Project: flink   File: TaskTestBase.java    License: Apache License 2.0 5 votes vote down vote up
public void registerFileOutputTask(
	FileOutputFormat<Record> outputFormat,
	String outPath,
	Configuration formatParams) {

	outputFormat.setOutputFilePath(new Path(outPath));
	outputFormat.setWriteMode(WriteMode.OVERWRITE);

	OperatorID operatorID = new OperatorID();
	new InputOutputFormatContainer(Thread.currentThread().getContextClassLoader())
		.addOutputFormat(operatorID, outputFormat)
		.addParameters(operatorID, formatParams)
		.write(new TaskConfig(this.mockEnv.getTaskConfiguration()));
}
 
Example 3
Source Project: flink   File: TaskTestBase.java    License: Apache License 2.0 5 votes vote down vote up
public void registerFileOutputTask(
	FileOutputFormat<Record> outputFormat,
	String outPath,
	Configuration formatParams) {

	outputFormat.setOutputFilePath(new Path(outPath));
	outputFormat.setWriteMode(WriteMode.OVERWRITE);

	OperatorID operatorID = new OperatorID();
	new InputOutputFormatContainer(Thread.currentThread().getContextClassLoader())
		.addOutputFormat(operatorID, outputFormat)
		.addParameters(operatorID, formatParams)
		.write(new TaskConfig(this.mockEnv.getTaskConfiguration()));
}
 
Example 4
Source Project: Flink-CEPplus   File: DataSet.java    License: Apache License 2.0 3 votes vote down vote up
/**
 * Writes a DataSet using a {@link FileOutputFormat} to a specified location.
 * This method adds a data sink to the program.
 *
 * @param outputFormat The FileOutputFormat to write the DataSet.
 * @param filePath The path to the location where the DataSet is written.
 * @param writeMode The mode of writing, indicating whether to overwrite existing files.
 * @return The DataSink that writes the DataSet.
 *
 * @see FileOutputFormat
 */
public DataSink<T> write(FileOutputFormat<T> outputFormat, String filePath, WriteMode writeMode) {
	Preconditions.checkNotNull(filePath, "File path must not be null.");
	Preconditions.checkNotNull(writeMode, "Write mode must not be null.");
	Preconditions.checkNotNull(outputFormat, "Output format must not be null.");

	outputFormat.setOutputFilePath(new Path(filePath));
	outputFormat.setWriteMode(writeMode);
	return output(outputFormat);
}
 
Example 5
Source Project: flink   File: DataSet.java    License: Apache License 2.0 3 votes vote down vote up
/**
 * Writes a DataSet using a {@link FileOutputFormat} to a specified location.
 * This method adds a data sink to the program.
 *
 * @param outputFormat The FileOutputFormat to write the DataSet.
 * @param filePath The path to the location where the DataSet is written.
 * @param writeMode The mode of writing, indicating whether to overwrite existing files.
 * @return The DataSink that writes the DataSet.
 *
 * @see FileOutputFormat
 */
public DataSink<T> write(FileOutputFormat<T> outputFormat, String filePath, WriteMode writeMode) {
	Preconditions.checkNotNull(filePath, "File path must not be null.");
	Preconditions.checkNotNull(writeMode, "Write mode must not be null.");
	Preconditions.checkNotNull(outputFormat, "Output format must not be null.");

	outputFormat.setOutputFilePath(new Path(filePath));
	outputFormat.setWriteMode(writeMode);
	return output(outputFormat);
}
 
Example 6
Source Project: flink   File: DataSet.java    License: Apache License 2.0 3 votes vote down vote up
/**
 * Writes a DataSet using a {@link FileOutputFormat} to a specified location.
 * This method adds a data sink to the program.
 *
 * @param outputFormat The FileOutputFormat to write the DataSet.
 * @param filePath The path to the location where the DataSet is written.
 * @param writeMode The mode of writing, indicating whether to overwrite existing files.
 * @return The DataSink that writes the DataSet.
 *
 * @see FileOutputFormat
 */
public DataSink<T> write(FileOutputFormat<T> outputFormat, String filePath, WriteMode writeMode) {
	Preconditions.checkNotNull(filePath, "File path must not be null.");
	Preconditions.checkNotNull(writeMode, "Write mode must not be null.");
	Preconditions.checkNotNull(outputFormat, "Output format must not be null.");

	outputFormat.setOutputFilePath(new Path(filePath));
	outputFormat.setWriteMode(writeMode);
	return output(outputFormat);
}