org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException Java Examples

The following examples show how to use org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: LeaderRetrievalUtils.java    From Flink-CEPplus with Apache License 2.0 6 votes vote down vote up
/**
 * Retrieves the current leader gateway using the given {@link LeaderRetrievalService}. If the
 * current leader could not be retrieved after the given timeout, then a
 * {@link LeaderRetrievalException} is thrown.
 *
 * @param leaderRetrievalService {@link LeaderRetrievalService} which is used for the leader retrieval
 * @param actorSystem ActorSystem which is used for the {@link LeaderRetrievalListener} implementation
 * @param timeout Timeout value for the retrieval call
 * @return The current leader gateway
 * @throws LeaderRetrievalException If the actor gateway could not be retrieved or the timeout has been exceeded
 */
public static ActorGateway retrieveLeaderGateway(
		LeaderRetrievalService leaderRetrievalService,
		ActorSystem actorSystem,
		FiniteDuration timeout)
	throws LeaderRetrievalException {
	LeaderGatewayListener listener = new LeaderGatewayListener(actorSystem, timeout);

	try {
		leaderRetrievalService.start(listener);

		Future<ActorGateway> actorGatewayFuture = listener.getActorGatewayFuture();

		return Await.result(actorGatewayFuture, timeout);
	} catch (Exception e) {
		throw new LeaderRetrievalException("Could not retrieve the leader gateway.", e);
	} finally {
		try {
			leaderRetrievalService.stop();
		} catch (Exception fe) {
			LOG.warn("Could not stop the leader retrieval service.", fe);
		}
	}
}
 
Example #2
Source File: TaskManagerRunner.java    From Flink-CEPplus with Apache License 2.0 6 votes vote down vote up
private static String determineTaskManagerBindAddressByConnectingToResourceManager(
		final Configuration configuration,
		final HighAvailabilityServices haServices) throws LeaderRetrievalException {

	final Time lookupTimeout = Time.milliseconds(AkkaUtils.getLookupTimeout(configuration).toMillis());

	final InetAddress taskManagerAddress = LeaderRetrievalUtils.findConnectingAddress(
		haServices.getResourceManagerLeaderRetriever(),
		lookupTimeout);

	LOG.info("TaskManager will use hostname/address '{}' ({}) for communication.",
		taskManagerAddress.getHostName(), taskManagerAddress.getHostAddress());

	HostBindPolicy bindPolicy = HostBindPolicy.fromString(configuration.getString(TaskManagerOptions.HOST_BIND_POLICY));
	return bindPolicy == HostBindPolicy.IP ? taskManagerAddress.getHostAddress() : taskManagerAddress.getHostName();
}
 
Example #3
Source File: LeaderRetrievalUtils.java    From Flink-CEPplus with Apache License 2.0 6 votes vote down vote up
/**
 * Retrieves the leader akka url and the current leader session ID. The values are stored in a
 * {@link LeaderConnectionInfo} instance.
 *
 * @param leaderRetrievalService Leader retrieval service to retrieve the leader connection
 *                               information
 * @param timeout Timeout when to give up looking for the leader
 * @return LeaderConnectionInfo containing the leader's akka URL and the current leader session
 * ID
 * @throws LeaderRetrievalException
 */
public static LeaderConnectionInfo retrieveLeaderConnectionInfo(
		LeaderRetrievalService leaderRetrievalService,
		FiniteDuration timeout
) throws LeaderRetrievalException {
	LeaderConnectionInfoListener listener = new LeaderConnectionInfoListener();

	try {
		leaderRetrievalService.start(listener);

		Future<LeaderConnectionInfo> connectionInfoFuture = listener.getLeaderConnectionInfoFuture();

		return Await.result(connectionInfoFuture, timeout);
	} catch (Exception e) {
		throw new LeaderRetrievalException("Could not retrieve the leader address and leader " +
				"session ID.", e);
	} finally {
		try {
			leaderRetrievalService.stop();
		} catch (Exception fe) {
			LOG.warn("Could not stop the leader retrieval service.", fe);
		}
	}
}
 
Example #4
Source File: LeaderRetrievalUtils.java    From flink with Apache License 2.0 6 votes vote down vote up
/**
 * Retrieves the leader akka url and the current leader session ID. The values are stored in a
 * {@link LeaderConnectionInfo} instance.
 *
 * @param leaderRetrievalService Leader retrieval service to retrieve the leader connection
 *                               information
 * @param timeout Timeout when to give up looking for the leader
 * @return LeaderConnectionInfo containing the leader's akka URL and the current leader session
 * ID
 * @throws LeaderRetrievalException
 */
public static LeaderConnectionInfo retrieveLeaderConnectionInfo(
		LeaderRetrievalService leaderRetrievalService,
		Duration timeout) throws LeaderRetrievalException {

	LeaderConnectionInfoListener listener = new LeaderConnectionInfoListener();

	try {
		leaderRetrievalService.start(listener);

		return listener.getLeaderConnectionInfoFuture().get(timeout.toMillis(), TimeUnit.MILLISECONDS);
	} catch (Exception e) {
		throw new LeaderRetrievalException("Could not retrieve the leader address and leader " +
			"session ID.", e);
	} finally {
		try {
			leaderRetrievalService.stop();
		} catch (Exception fe) {
			LOG.warn("Could not stop the leader retrieval service.", fe);
		}
	}
}
 
Example #5
Source File: LeaderRetrievalUtils.java    From Flink-CEPplus with Apache License 2.0 6 votes vote down vote up
public static InetAddress findConnectingAddress(
		LeaderRetrievalService leaderRetrievalService,
		FiniteDuration timeout) throws LeaderRetrievalException {
	ConnectionUtils.LeaderConnectingAddressListener listener = new ConnectionUtils.LeaderConnectingAddressListener();

	try {
		leaderRetrievalService.start(listener);

		LOG.info("Trying to select the network interface and address to use " +
				"by connecting to the leading JobManager.");

		LOG.info("TaskManager will try to connect for " + timeout +
				" before falling back to heuristics");

		return listener.findConnectingAddress(timeout);
	} catch (Exception e) {
		throw new LeaderRetrievalException("Could not find the connecting address by " +
				"connecting to the current leader.", e);
	} finally {
		try {
			leaderRetrievalService.stop();
		} catch (Exception fe) {
			LOG.warn("Could not stop the leader retrieval service.", fe);
		}
	}
}
 
Example #6
Source File: TaskManagerRunner.java    From flink with Apache License 2.0 6 votes vote down vote up
private static String determineTaskManagerBindAddressByConnectingToResourceManager(
		final Configuration configuration,
		final HighAvailabilityServices haServices) throws LeaderRetrievalException {

	final Duration lookupTimeout = AkkaUtils.getLookupTimeout(configuration);

	final InetAddress taskManagerAddress = LeaderRetrievalUtils.findConnectingAddress(
		haServices.getResourceManagerLeaderRetriever(),
		lookupTimeout);

	LOG.info("TaskManager will use hostname/address '{}' ({}) for communication.",
		taskManagerAddress.getHostName(), taskManagerAddress.getHostAddress());

	HostBindPolicy bindPolicy = HostBindPolicy.fromString(configuration.getString(TaskManagerOptions.HOST_BIND_POLICY));
	return bindPolicy == HostBindPolicy.IP ? taskManagerAddress.getHostAddress() : taskManagerAddress.getHostName();
}
 
Example #7
Source File: LeaderRetrievalUtils.java    From flink with Apache License 2.0 6 votes vote down vote up
public static InetAddress findConnectingAddress(
		LeaderRetrievalService leaderRetrievalService,
		FiniteDuration timeout) throws LeaderRetrievalException {
	ConnectionUtils.LeaderConnectingAddressListener listener = new ConnectionUtils.LeaderConnectingAddressListener();

	try {
		leaderRetrievalService.start(listener);

		LOG.info("Trying to select the network interface and address to use " +
				"by connecting to the leading JobManager.");

		LOG.info("TaskManager will try to connect for " + timeout +
				" before falling back to heuristics");

		return listener.findConnectingAddress(timeout);
	} catch (Exception e) {
		throw new LeaderRetrievalException("Could not find the connecting address by " +
				"connecting to the current leader.", e);
	} finally {
		try {
			leaderRetrievalService.stop();
		} catch (Exception fe) {
			LOG.warn("Could not stop the leader retrieval service.", fe);
		}
	}
}
 
Example #8
Source File: TaskManagerRunner.java    From flink with Apache License 2.0 6 votes vote down vote up
private static String determineTaskManagerBindAddressByConnectingToResourceManager(
		final Configuration configuration,
		final HighAvailabilityServices haServices) throws LeaderRetrievalException {

	final Time lookupTimeout = Time.milliseconds(AkkaUtils.getLookupTimeout(configuration).toMillis());

	final InetAddress taskManagerAddress = LeaderRetrievalUtils.findConnectingAddress(
		haServices.getResourceManagerLeaderRetriever(),
		lookupTimeout);

	LOG.info("TaskManager will use hostname/address '{}' ({}) for communication.",
		taskManagerAddress.getHostName(), taskManagerAddress.getHostAddress());

	HostBindPolicy bindPolicy = HostBindPolicy.fromString(configuration.getString(TaskManagerOptions.HOST_BIND_POLICY));
	return bindPolicy == HostBindPolicy.IP ? taskManagerAddress.getHostAddress() : taskManagerAddress.getHostName();
}
 
Example #9
Source File: LeaderRetrievalUtils.java    From flink with Apache License 2.0 6 votes vote down vote up
/**
 * Retrieves the leader akka url and the current leader session ID. The values are stored in a
 * {@link LeaderConnectionInfo} instance.
 *
 * @param leaderRetrievalService Leader retrieval service to retrieve the leader connection
 *                               information
 * @param timeout Timeout when to give up looking for the leader
 * @return LeaderConnectionInfo containing the leader's akka URL and the current leader session
 * ID
 * @throws LeaderRetrievalException
 */
public static LeaderConnectionInfo retrieveLeaderConnectionInfo(
		LeaderRetrievalService leaderRetrievalService,
		FiniteDuration timeout
) throws LeaderRetrievalException {
	LeaderConnectionInfoListener listener = new LeaderConnectionInfoListener();

	try {
		leaderRetrievalService.start(listener);

		Future<LeaderConnectionInfo> connectionInfoFuture = listener.getLeaderConnectionInfoFuture();

		return Await.result(connectionInfoFuture, timeout);
	} catch (Exception e) {
		throw new LeaderRetrievalException("Could not retrieve the leader address and leader " +
				"session ID.", e);
	} finally {
		try {
			leaderRetrievalService.stop();
		} catch (Exception fe) {
			LOG.warn("Could not stop the leader retrieval service.", fe);
		}
	}
}
 
Example #10
Source File: LeaderRetrievalUtils.java    From flink with Apache License 2.0 5 votes vote down vote up
public static InetAddress findConnectingAddress(
		LeaderRetrievalService leaderRetrievalService,
		Duration timeout) throws LeaderRetrievalException {

	ConnectionUtils.LeaderConnectingAddressListener listener = new ConnectionUtils.LeaderConnectingAddressListener();

	try {
		leaderRetrievalService.start(listener);

		LOG.info("Trying to select the network interface and address to use " +
			"by connecting to the leading JobManager.");

		LOG.info("TaskManager will try to connect for " + timeout +
			" before falling back to heuristics");

		return listener.findConnectingAddress(timeout);
	} catch (Exception e) {
		throw new LeaderRetrievalException("Could not find the connecting address by " +
			"connecting to the current leader.", e);
	} finally {
		try {
			leaderRetrievalService.stop();
		} catch (Exception fe) {
			LOG.warn("Could not stop the leader retrieval service.", fe);
		}
	}
}
 
Example #11
Source File: ClusterClient.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up
/**
 * Creates a new ActorSystem or returns an existing one.
 * @return ActorSystem
 * @throws Exception if the ActorSystem could not be created
 */
@Override
public ActorSystem get() throws FlinkException {

	if (!isLoaded()) {
		// start actor system
		log.info("Starting client actor system.");

		final InetAddress ownHostname;
		try {
			ownHostname = LeaderRetrievalUtils.findConnectingAddress(
				highAvailabilityServices.getJobManagerLeaderRetriever(HighAvailabilityServices.DEFAULT_JOB_ID),
				timeout);
		} catch (LeaderRetrievalException lre) {
			throw new FlinkException("Could not find out our own hostname by connecting to the " +
				"leading JobManager. Please make sure that the Flink cluster has been started.", lre);
		}

		try {
			actorSystem = BootstrapTools.startActorSystem(
				configuration,
				ownHostname.getCanonicalHostName(),
				0,
				log);
		} catch (Exception e) {
			throw new FlinkException("Could not start the ActorSystem lazily.", e);
		}
	}

	return actorSystem;
}
 
Example #12
Source File: ClusterClient.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up
/**
 * Returns the {@link ActorGateway} of the current job manager leader using
 * the {@link LeaderRetrievalService}.
 *
 * @return ActorGateway of the current job manager leader
 * @throws Exception
 */
public ActorGateway getJobManagerGateway() throws FlinkException {
	log.debug("Looking up JobManager");

	try {
		return LeaderRetrievalUtils.retrieveLeaderGateway(
			highAvailabilityServices.getJobManagerLeaderRetriever(HighAvailabilityServices.DEFAULT_JOB_ID),
			actorSystemLoader.get(),
			lookupTimeout);
	} catch (LeaderRetrievalException lre) {
		throw new FlinkException("Could not connect to the leading JobManager. Please check that the " +
			"JobManager is running.", lre);
	}
}
 
Example #13
Source File: ConnectionUtils.java    From flink with Apache License 2.0 4 votes vote down vote up
public InetAddress findConnectingAddress(
		Duration timeout) throws LeaderRetrievalException {
	return findConnectingAddress(timeout, defaultLoggingDelay);
}
 
Example #14
Source File: ConnectionUtils.java    From flink with Apache License 2.0 4 votes vote down vote up
public InetAddress findConnectingAddress(
		FiniteDuration timeout) throws LeaderRetrievalException {
	return findConnectingAddress(timeout, defaultLoggingDelay);
}
 
Example #15
Source File: LeaderRetrievalUtils.java    From flink with Apache License 2.0 4 votes vote down vote up
public static InetAddress findConnectingAddress(
		LeaderRetrievalService leaderRetrievalService,
		Time timeout) throws LeaderRetrievalException {
	return findConnectingAddress(leaderRetrievalService, new FiniteDuration(timeout.getSize(), timeout.getUnit()));
}
 
Example #16
Source File: MiniClusterClient.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up
@Override
public LeaderConnectionInfo getClusterConnectionInfo() throws LeaderRetrievalException {
	return LeaderRetrievalUtils.retrieveLeaderConnectionInfo(
		highAvailabilityServices.getDispatcherLeaderRetriever(),
		timeout);
}
 
Example #17
Source File: RestClusterClient.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up
@Override
public LeaderConnectionInfo getClusterConnectionInfo() throws LeaderRetrievalException {
	return LeaderRetrievalUtils.retrieveLeaderConnectionInfo(
		highAvailabilityServices.getDispatcherLeaderRetriever(),
		timeout);
}
 
Example #18
Source File: ConnectionUtils.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up
public InetAddress findConnectingAddress(
		FiniteDuration timeout) throws LeaderRetrievalException {
	return findConnectingAddress(timeout, defaultLoggingDelay);
}
 
Example #19
Source File: LeaderRetrievalUtils.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up
public static InetAddress findConnectingAddress(
		LeaderRetrievalService leaderRetrievalService,
		Time timeout) throws LeaderRetrievalException {
	return findConnectingAddress(leaderRetrievalService, new FiniteDuration(timeout.getSize(), timeout.getUnit()));
}
 
Example #20
Source File: LeaderRetrievalUtils.java    From flink with Apache License 2.0 2 votes vote down vote up
/**
 * Retrieves the leader akka url and the current leader session ID. The values are stored in a
 * {@link LeaderConnectionInfo} instance.
 *
 * @param leaderRetrievalService Leader retrieval service to retrieve the leader connection
 *                               information
 * @param timeout Timeout when to give up looking for the leader
 * @return LeaderConnectionInfo containing the leader's akka URL and the current leader session
 * ID
 * @throws LeaderRetrievalException
 */
public static LeaderConnectionInfo retrieveLeaderConnectionInfo(
		LeaderRetrievalService leaderRetrievalService,
		Time timeout) throws LeaderRetrievalException {
	return retrieveLeaderConnectionInfo(leaderRetrievalService, FutureUtils.toFiniteDuration(timeout));
}
 
Example #21
Source File: ClusterClient.java    From Flink-CEPplus with Apache License 2.0 2 votes vote down vote up
/**
 * Gets the current cluster connection info (may change in case of a HA setup).
 *
 * @return The the connection info to the leader component of the cluster
 * @throws LeaderRetrievalException if the leader could not be retrieved
 */
public LeaderConnectionInfo getClusterConnectionInfo() throws LeaderRetrievalException {
	return LeaderRetrievalUtils.retrieveLeaderConnectionInfo(
		highAvailabilityServices.getJobManagerLeaderRetriever(HighAvailabilityServices.DEFAULT_JOB_ID),
		timeout);
}
 
Example #22
Source File: ClusterClient.java    From flink with Apache License 2.0 2 votes vote down vote up
/**
 * Gets the current cluster connection info (may change in case of a HA setup).
 *
 * @return The the connection info to the leader component of the cluster
 * @throws LeaderRetrievalException if the leader could not be retrieved
 */
public LeaderConnectionInfo getClusterConnectionInfo() throws LeaderRetrievalException {
	return LeaderRetrievalUtils.retrieveLeaderConnectionInfo(
		highAvailabilityServices.getDispatcherLeaderRetriever(),
		timeout);
}
 
Example #23
Source File: LeaderRetrievalUtils.java    From Flink-CEPplus with Apache License 2.0 2 votes vote down vote up
/**
 * Retrieves the current leader session id of the component identified by the given leader
 * retrieval service.
 *
 * @param leaderRetrievalService Leader retrieval service to be used for the leader retrieval
 * @param timeout Timeout for the leader retrieval
 * @return The leader session id of the retrieved leader
 * @throws LeaderRetrievalException if the leader retrieval operation fails (including a timeout)
 */
public static UUID retrieveLeaderSessionId(
		LeaderRetrievalService leaderRetrievalService,
		FiniteDuration timeout) throws LeaderRetrievalException {
	return retrieveLeaderConnectionInfo(leaderRetrievalService, timeout).getLeaderSessionID();
}
 
Example #24
Source File: LeaderRetrievalUtils.java    From Flink-CEPplus with Apache License 2.0 2 votes vote down vote up
/**
 * Retrieves the leader akka url and the current leader session ID. The values are stored in a
 * {@link LeaderConnectionInfo} instance.
 *
 * @param leaderRetrievalService Leader retrieval service to retrieve the leader connection
 *                               information
 * @param timeout Timeout when to give up looking for the leader
 * @return LeaderConnectionInfo containing the leader's akka URL and the current leader session
 * ID
 * @throws LeaderRetrievalException
 */
public static LeaderConnectionInfo retrieveLeaderConnectionInfo(
		LeaderRetrievalService leaderRetrievalService,
		Time timeout) throws LeaderRetrievalException {
	return retrieveLeaderConnectionInfo(leaderRetrievalService, FutureUtils.toFiniteDuration(timeout));
}