Java Code Examples for com.amazonaws.client.builder.AwsClientBuilder#EndpointConfiguration

The following examples show how to use com.amazonaws.client.builder.AwsClientBuilder#EndpointConfiguration . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: XRayClient.java    From aws-xray-sdk-java with Apache License 2.0 6 votes vote down vote up
/**
 *
 * @deprecated aws-xray-recorder only supports communicating with the X-Ray daemon, which does
 * not require the usual AWS API signatures so we have stopped using the SDK X-Ray client.
 */
@Deprecated
public static AWSXRay newClient() {
    DaemonConfiguration config = new DaemonConfiguration();

    ClientConfiguration clientConfig = new ClientConfiguration()
            .withRequestTimeout(TIME_OUT);

    AwsClientBuilder.EndpointConfiguration endpointConfig = new AwsClientBuilder.EndpointConfiguration(
            config.getEndpointForTCPConnection(),
            DUMMY_REGION
    );

    return AWSXRayClientBuilder.standard()
            .withEndpointConfiguration(endpointConfig)
            .withClientConfiguration(clientConfig)
            .withCredentials(ANONYMOUS_CREDENTIALS) // This will entirely skip signing too
            .build();

}
 
Example 2
Source File: DynamoManager.java    From dynamo-cassandra-proxy with Apache License 2.0 5 votes vote down vote up
public synchronized AmazonDynamoDB get() {
    if (ddb == null) {
        AwsClientBuilder.EndpointConfiguration endpointConfiguration = new AwsClientBuilder.EndpointConfiguration(dynamodbEndpoint, signinRegion);
        ddb = AmazonDynamoDBClientBuilder.standard().withEndpointConfiguration(endpointConfiguration).build();
    }

    return ddb;
}
 
Example 3
Source File: Aws1ITest.java    From java-specialagent with Apache License 2.0 5 votes vote down vote up
private static AmazonDynamoDB buildClient() {
  final AwsClientBuilder.EndpointConfiguration endpointConfiguration = new AwsClientBuilder.EndpointConfiguration("http://localhost:8000", "us-west-2");
  final BasicAWSCredentials awsCreds = new BasicAWSCredentials("access_key_id", "secret_key_id");
  final AmazonDynamoDBClientBuilder builder = AmazonDynamoDBClientBuilder
    .standard()
    .withEndpointConfiguration(endpointConfiguration)
    .withCredentials(new AWSStaticCredentialsProvider(awsCreds))
    .withClientConfiguration(new ClientConfiguration().withConnectionTimeout(1));
  return builder.build();
}
 
Example 4
Source File: S3UploadClient.java    From ambari-logsearch with Apache License 2.0 5 votes vote down vote up
@Override
public void init(LogFeederProps logFeederProps) {
  SecretKeyPair keyPair = getSecretKeyPair(logFeederProps, s3OutputConfig);
  AWSCredentials awsCredentials = new BasicAWSCredentials(new String(keyPair.getAccessKey()), new String(keyPair.getSecretKey()));
  AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(awsCredentials);
  AwsClientBuilder.EndpointConfiguration endpointConf = new AwsClientBuilder.EndpointConfiguration(s3OutputConfig.getEndpoint(), s3OutputConfig.getRegion());
  s3Client = AmazonS3ClientBuilder.standard()
    .withCredentials(credentialsProvider)
    .withEndpointConfiguration(endpointConf)
    .withPathStyleAccessEnabled(s3OutputConfig.isPathStyleAccess())
    .build();
  bootstrapBucket(s3OutputConfig.getBucketConfig().getBucket(), s3OutputConfig.getBucketConfig());
}
 
Example 5
Source File: HaystackKinesisForwarderTest.java    From pitchfork with Apache License 2.0 5 votes vote down vote up
private static void setKinesisServiceEndpoint() {
    // https://github.com/localstack/localstack/blob/e479afa41df908305c4177276237925accc77e10/localstack/ext/java/src/test/java/cloud/localstack/BasicFunctionalityTest.java#L54
    System.setProperty("com.amazonaws.sdk.disableCbor", "true");

    var serviceEndpoint = kinesisContainer.getEndpointConfiguration(KINESIS).getServiceEndpoint();
    var endpointConfiguration = new AwsClientBuilder.EndpointConfiguration(serviceEndpoint, "us-west-1");

    KINESIS_SERVICE_ENDPOINT = endpointConfiguration.getServiceEndpoint();
}
 
Example 6
Source File: MockedS3ClientTest.java    From lightning with MIT License 5 votes vote down vote up
@BeforeClass
public void setupEnv() {
    int port = 8001;

    AwsClientBuilder.EndpointConfiguration endpointConfiguration = new AwsClientBuilder.EndpointConfiguration(S3_MOCK_URL, REGION);
    amazonS3Client = AmazonS3ClientBuilder
            .standard()
            .withPathStyleAccessEnabled(true)
            .withEndpointConfiguration(endpointConfiguration)
            .build();

    s3Mock = new S3Mock.Builder().withPort(port).withInMemoryBackend().build();
    s3Mock.start();
}
 
Example 7
Source File: CommandLineInterfaceTests.java    From dynamodb-cross-region-library with Apache License 2.0 5 votes vote down vote up
@Test
public void testCreateEndpointConfiguration() {
    final AwsClientBuilder.EndpointConfiguration config = CommandLineInterface.createEndpointConfiguration(RegionUtils.getRegion(Regions.US_EAST_1.getName()),
            Optional.<String>absent(), AmazonDynamoDB.ENDPOINT_PREFIX);
    assertTrue(config.getServiceEndpoint().contains("https"));
    assertTrue(config.getServiceEndpoint().contains(AmazonDynamoDB.ENDPOINT_PREFIX));
    assertEquals(Regions.US_EAST_1.getName(), config.getSigningRegion());
}
 
Example 8
Source File: DynamoDbDelegate.java    From dynamodb-janusgraph-storage-backend with Apache License 2.0 5 votes vote down vote up
@VisibleForTesting
static AwsClientBuilder.EndpointConfiguration getEndpointConfiguration(final Optional<String> endpoint, final String signingRegion) {
    Preconditions.checkArgument(endpoint != null, "must provide an optional endpoint and not null");
    Preconditions.checkArgument(!Strings.isNullOrEmpty(signingRegion), "must provide a signing region");
    final String expectedServiceEndpoint = "https://" + Region.getRegion(Regions.fromName(signingRegion)).getServiceEndpoint(AmazonDynamoDB.ENDPOINT_PREFIX);
    if (endpoint.isPresent() && !Strings.isNullOrEmpty(endpoint.get())) {
        final String regionParsedFromEndpoint = AwsHostNameUtils.parseRegion(endpoint.get(), AmazonDynamoDB.ENDPOINT_PREFIX);
        Preconditions.checkArgument(regionParsedFromEndpoint == null || signingRegion.equals(regionParsedFromEndpoint));
        return new AwsClientBuilder.EndpointConfiguration(endpoint.get(), signingRegion);
    } else {
        //Regions.fromName will throw IllegalArgumentException if signingRegion is not valid.
        return new AwsClientBuilder.EndpointConfiguration(expectedServiceEndpoint, signingRegion);
    }
}
 
Example 9
Source File: AWSLambdaConfiguration.java    From micronaut-aws with Apache License 2.0 5 votes vote down vote up
/**
 * @param endpointConfiguration The {@link AwsClientBuilder#getEndpoint()}
 */
@Inject
public void setEndpointConfiguration(@Nullable AwsClientBuilder.EndpointConfiguration endpointConfiguration) {
    if (endpointConfiguration != null) {
        builder.setEndpointConfiguration(endpointConfiguration);
    }
}
 
Example 10
Source File: DynamoDbDelegateTest.java    From dynamodb-janusgraph-storage-backend with Apache License 2.0 4 votes vote down vote up
@Test
public void getEndpointConfiguration_whenEndpointEmptyAndRegionValid_returnConfig() {
    final AwsClientBuilder.EndpointConfiguration config = DynamoDbDelegate.getEndpointConfiguration(EMPTY_ENDPOINT, VALID_REGION);
    assertEquals(VALID_REGION, config.getSigningRegion());
    assertTrue(config.getServiceEndpoint().contains(VALID_REGION));
}
 
Example 11
Source File: CommandLineInterface.java    From dynamodb-cross-region-library with Apache License 2.0 4 votes vote down vote up
@VisibleForTesting
static AwsClientBuilder.EndpointConfiguration createEndpointConfiguration(Region region, Optional<String> endpoint, String endpointPrefix) {
    return new AwsClientBuilder.EndpointConfiguration(endpoint.or("https://" + region.getServiceEndpoint(endpointPrefix)), region.getName());
}
 
Example 12
Source File: DynamoDbDelegateTest.java    From dynamodb-janusgraph-storage-backend with Apache License 2.0 4 votes vote down vote up
@Test
public void getEndpointConfiguration_whenEndpointValidServiceAndRegionValid_returnsConfig() {
    final AwsClientBuilder.EndpointConfiguration config = DynamoDbDelegate.getEndpointConfiguration(VALID_DYNAMODB_ENDPOINT, VALID_REGION);
    assertEquals(VALID_REGION, config.getSigningRegion());
    assertEquals(VALID_DYNAMODB_ENDPOINT.get(), config.getServiceEndpoint());
}
 
Example 13
Source File: DynamoDbDelegateTest.java    From dynamodb-janusgraph-storage-backend with Apache License 2.0 4 votes vote down vote up
@Test
public void getEndpointConfiguration_whenEndpointValidLocalAndRegionValid_returnsConfig() {
    final AwsClientBuilder.EndpointConfiguration config = DynamoDbDelegate.getEndpointConfiguration(VALID_DYNAMODB_LOCAL_ENDPOINT, VALID_REGION);
    assertEquals(VALID_REGION, config.getSigningRegion());
    assertEquals(VALID_DYNAMODB_LOCAL_ENDPOINT.get(), config.getServiceEndpoint());
}
 
Example 14
Source File: DynamoDbDelegateTest.java    From dynamodb-janusgraph-storage-backend with Apache License 2.0 4 votes vote down vote up
@Test
public void getEndpointConfiguration_whenEndpointValidNotAUrlAndRegionValid_returnConfig() {
    final AwsClientBuilder.EndpointConfiguration config = DynamoDbDelegate.getEndpointConfiguration(VALID_NOT_A_URL_ENDPOINT, VALID_REGION);
    assertEquals(VALID_REGION, config.getSigningRegion());
    assertEquals(VALID_NOT_A_URL_ENDPOINT.get(), config.getServiceEndpoint());
}
 
Example 15
Source File: DynamoDbDelegateTest.java    From dynamodb-janusgraph-storage-backend with Apache License 2.0 4 votes vote down vote up
@Test
public void getEndpointConfiguration_whenEndpointValidEmptyStringAndRegionValid_throwIllegalArgumentException() {
    final AwsClientBuilder.EndpointConfiguration config = DynamoDbDelegate.getEndpointConfiguration(VALID_EMPTY_STRING_ENDPOINT, VALID_REGION);
    assertEquals(VALID_REGION, config.getSigningRegion());
    assertEquals("https://dynamodb." + VALID_REGION + ".amazonaws.com", config.getServiceEndpoint());
}
 
Example 16
Source File: ExtendedDockerTestUtils.java    From spring-cloud-stream-binder-aws-kinesis with Apache License 2.0 4 votes vote down vote up
private static AwsClientBuilder.EndpointConfiguration createEndpointConfiguration(Supplier<String> supplier) {
	return new AwsClientBuilder.EndpointConfiguration(supplier.get(), TestUtils.DEFAULT_REGION);
}
 
Example 17
Source File: CommandLineInterface.java    From dynamodb-cross-region-library with Apache License 2.0 4 votes vote down vote up
public Worker createWorker() {

        // use default credential provider chain to locate appropriate credentials
        final AWSCredentialsProvider credentialsProvider = new DefaultAWSCredentialsProviderChain();

        // initialize DynamoDB client and set the endpoint properly for source table / region
        final AmazonDynamoDB dynamodbClient = AmazonDynamoDBClientBuilder.standard()
                .withCredentials(credentialsProvider)
                .withEndpointConfiguration(createEndpointConfiguration(sourceRegion, sourceDynamodbEndpoint, AmazonDynamoDB.ENDPOINT_PREFIX))
                .build();

        // initialize Streams client
        final AwsClientBuilder.EndpointConfiguration streamsEndpointConfiguration = createEndpointConfiguration(sourceRegion,
                sourceDynamodbStreamsEndpoint, AmazonDynamoDBStreams.ENDPOINT_PREFIX);
        final ClientConfiguration streamsClientConfig = new ClientConfiguration().withGzip(false);
        final AmazonDynamoDBStreams streamsClient = AmazonDynamoDBStreamsClientBuilder.standard()
                .withCredentials(credentialsProvider)
                .withEndpointConfiguration(streamsEndpointConfiguration)
                .withClientConfiguration(streamsClientConfig)
                .build();

        // obtain the Stream ID associated with the source table
        final String streamArn = dynamodbClient.describeTable(sourceTable).getTable().getLatestStreamArn();
        final boolean streamEnabled = DynamoDBConnectorUtilities.isStreamsEnabled(streamsClient, streamArn, DynamoDBConnectorConstants.NEW_AND_OLD);
        Preconditions.checkArgument(streamArn != null, DynamoDBConnectorConstants.MSG_NO_STREAMS_FOUND);
        Preconditions.checkState(streamEnabled, DynamoDBConnectorConstants.STREAM_NOT_READY);

        // initialize DynamoDB client for KCL
        final AmazonDynamoDB kclDynamoDBClient = AmazonDynamoDBClientBuilder.standard()
                .withCredentials(credentialsProvider)
                .withEndpointConfiguration(createKclDynamoDbEndpointConfiguration())
                .build();

        // initialize DynamoDB Streams Adapter client and set the Streams endpoint properly
        final AmazonDynamoDBStreamsAdapterClient streamsAdapterClient = new AmazonDynamoDBStreamsAdapterClient(streamsClient);

        // initialize CloudWatch client and set the region to emit metrics to
        final AmazonCloudWatch kclCloudWatchClient;
        if (isPublishCloudWatch) {
            kclCloudWatchClient = AmazonCloudWatchClientBuilder.standard()
                    .withCredentials(credentialsProvider)
                    .withRegion(kclRegion.or(sourceRegion).getName()).build();
        } else {
            kclCloudWatchClient = new NoopCloudWatch();
        }

        // try to get taskname from command line arguments, auto generate one if needed
        final AwsClientBuilder.EndpointConfiguration destinationEndpointConfiguration = createEndpointConfiguration(destinationRegion,
                destinationDynamodbEndpoint, AmazonDynamoDB.ENDPOINT_PREFIX);
        final String actualTaskName = DynamoDBConnectorUtilities.getTaskName(sourceRegion, destinationRegion, taskName, sourceTable, destinationTable);

        // set the appropriate Connector properties for the destination KCL configuration
        final Properties properties = new Properties();
        properties.put(DynamoDBStreamsConnectorConfiguration.PROP_APP_NAME, actualTaskName);
        properties.put(DynamoDBStreamsConnectorConfiguration.PROP_DYNAMODB_ENDPOINT, destinationEndpointConfiguration.getServiceEndpoint());
        properties.put(DynamoDBStreamsConnectorConfiguration.PROP_DYNAMODB_DATA_TABLE_NAME, destinationTable);
        properties.put(DynamoDBStreamsConnectorConfiguration.PROP_REGION_NAME, destinationRegion.getName());

        // create the record processor factory based on given pipeline and connector configurations
        // use the master to replicas pipeline
        final KinesisConnectorRecordProcessorFactory<Record, Record> factory = new KinesisConnectorRecordProcessorFactory<>(
                new DynamoDBMasterToReplicasPipeline(), new DynamoDBStreamsConnectorConfiguration(properties, credentialsProvider));

        // create the KCL configuration with default values
        final KinesisClientLibConfiguration kclConfig = new KinesisClientLibConfiguration(actualTaskName,
                streamArn,
                credentialsProvider,
                DynamoDBConnectorConstants.WORKER_LABEL + actualTaskName + UUID.randomUUID().toString())
                // worker will use checkpoint table if available, otherwise it is safer
                // to start at beginning of the stream
                .withInitialPositionInStream(InitialPositionInStream.TRIM_HORIZON)
                // we want the maximum batch size to avoid network transfer latency overhead
                .withMaxRecords(getRecordsLimit.or(DynamoDBConnectorConstants.STREAMS_RECORDS_LIMIT))
                // wait a reasonable amount of time - default 0.5 seconds
                .withIdleTimeBetweenReadsInMillis(DynamoDBConnectorConstants.IDLE_TIME_BETWEEN_READS)
                // Remove calls to GetShardIterator
                .withValidateSequenceNumberBeforeCheckpointing(false)
                // make parent shard poll interval tunable to decrease time to run integration test
                .withParentShardPollIntervalMillis(parentShardPollIntervalMillis.or(DynamoDBConnectorConstants.DEFAULT_PARENT_SHARD_POLL_INTERVAL_MILLIS))
                // avoid losing leases too often - default 60 seconds
                .withFailoverTimeMillis(DynamoDBConnectorConstants.KCL_FAILOVER_TIME);

        // create the KCL worker for this connector
        return new Worker(factory, kclConfig, streamsAdapterClient, kclDynamoDBClient, kclCloudWatchClient);
    }
 
Example 18
Source File: DynamoStreamsManager.java    From dynamo-cassandra-proxy with Apache License 2.0 4 votes vote down vote up
public void configure(DCProxyConfiguration config) {

        //TODO make table name dynamic
        String tableName = "test";

        this.dynamodbEndpoint = config.getAwsDynamodbEndpoint();
        this.streamsEndpoint = config.getStreamsEndpoint();
        this.signinRegion = config.getDynamoRegion();
        this.accessKey = config.getDynamoAccessKey();
        this.secretKey = config.getDynamoSecretKey();

        Properties props = System.getProperties();
        props.setProperty("aws.accessKeyId", accessKey);
        props.setProperty("aws.secretKey", secretKey);

        AwsClientBuilder.EndpointConfiguration endpointConfiguration =
                new AwsClientBuilder.EndpointConfiguration(streamsEndpoint, signinRegion);
        SystemPropertiesCredentialsProvider spcp = new SystemPropertiesCredentialsProvider();

        realDDB = AmazonDynamoDBClientBuilder.standard().
                withRegion(Regions.US_EAST_2).
                //withEndpointConfiguration(endpointConfiguration).
                withCredentials(spcp).build();

        DescribeTableResult tableResult = realDDB.describeTable(tableName);
        streamArn = tableResult.getTable().getLatestStreamArn();
        //streamSpec = tableResult.getTable().getStreamSpecification();
        streamsClient = AmazonDynamoDBStreamsClientBuilder.standard().withEndpointConfiguration(endpointConfiguration).build();

        adapterClient = new AmazonDynamoDBStreamsAdapterClient(streamsClient);

        recordProcessorFactory = new StreamsRecordProcessorFactory(ddbProxy, tableName);

        workerConfig = new KinesisClientLibConfiguration("test-app",
                streamArn,
                spcp,
                "streams-worker")
                .withMaxRecords(1000)
                .withIdleTimeBetweenReadsInMillis(500)
                .withInitialPositionInStream(InitialPositionInStream.TRIM_HORIZON);
        AmazonCloudWatch cloudWatchClient;
        cloudWatchClient = AmazonCloudWatchClientBuilder.standard()
        .withRegion(signinRegion)
        .build();

        System.out.println("Creating worker for stream: " + streamArn);

        /*
        DescribeStreamRequest request = new DescribeStreamRequest();
        DescribeStreamRequestAdapter describeStreamResult = new DescribeStreamRequestAdapter(request);
        String id = describeStreamResult.getExclusiveStartShardId();
        String id2 = describeStreamResult.withStreamArn(streamArn).getExclusiveStartShardId();
        */

        Worker worker = StreamsWorkerFactory.createDynamoDbStreamsWorker(
                recordProcessorFactory,
                workerConfig,
                adapterClient,
                realDDB,
                cloudWatchClient
        );

        System.out.println("Starting worker...");
        Thread t = new Thread(worker);
        t.start();
    }
 
Example 19
Source File: LocalStackContainer.java    From testcontainers-java with MIT License 2 votes vote down vote up
/**
 * Provides an endpoint configuration that is preconfigured to communicate with a given simulated service.
 * The provided endpoint configuration should be set in the AWS Java SDK when building a client, e.g.:
 * <pre><code>AmazonS3 s3 = AmazonS3ClientBuilder
        .standard()
        .withEndpointConfiguration(localstack.getEndpointConfiguration(S3))
        .withCredentials(localstack.getDefaultCredentialsProvider())
        .build();
 </code></pre>
 * or for AWS SDK v2
 * <pre><code>S3Client s3 = S3Client
         .builder()
         .endpointOverride(localstack.getEndpointOverride(LocalStackContainer.Service.S3))
         .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(
            localstack.getAccessKey(), localstack.getSecretKey()
         )))
         .region(Region.of(localstack.getRegion()))
         .build()
 </code></pre>
 * <p><strong>Please note that this method is only intended to be used for configuring AWS SDK clients
 * that are running on the test host. If other containers need to call this one, they should be configured
 * specifically to do so using a Docker network and appropriate addressing.</strong></p>
 *
 * @param service the service that is to be accessed
 * @return an {@link AwsClientBuilder.EndpointConfiguration}
 */
public AwsClientBuilder.EndpointConfiguration getEndpointConfiguration(Service service) {
    return new AwsClientBuilder.EndpointConfiguration(getEndpointOverride(service).toString(), getRegion());
}
 
Example 20
Source File: DynaliteContainer.java    From testcontainers-java with MIT License 2 votes vote down vote up
/**
 * Gets {@link AwsClientBuilder.EndpointConfiguration}
 * that may be used to connect to this container.
 *
 * @return endpoint configuration
 */
public AwsClientBuilder.EndpointConfiguration getEndpointConfiguration() {
    return new AwsClientBuilder.EndpointConfiguration("http://" +
            this.getHost() + ":" +
            this.getMappedPort(MAPPED_PORT), null);
}