com.google.cloud.storage.contrib.nio.CloudStorageConfiguration Java Examples

The following examples show how to use com.google.cloud.storage.contrib.nio.CloudStorageConfiguration. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: GoogleStorageUtils.java    From picard with MIT License 5 votes vote down vote up 
/** The config we want to use. **/
private static CloudStorageConfiguration getCloudStorageConfiguration(int maxReopens, String requesterProject) {
    CloudStorageConfiguration.Builder builder = CloudStorageConfiguration.builder()
            // if the channel errors out, re-open up to this many times
            .maxChannelReopens(maxReopens);
    if (!Strings.isNullOrEmpty(requesterProject)) {
        // enable requester pays and indicate who pays
        builder = builder.autoDetectRequesterPays(true).userProject(requesterProject);
    }

    // this causes the gcs filesystem to treat files that end in a / as a directory
    // true is the default but this protects against future changes in behavior
    builder.usePseudoDirectories(true);
    return builder.build();
}
Example #2
Source File: BucketUtils.java    From gatk with BSD 3-Clause "New" or "Revised" License 5 votes vote down vote up 
/**
 * The config we want to use.
 *
 * @param maxReopens If the GCS bucket channel errors out, how many times it will attempt to
 *                   re-initiate the connection.
 * @param requesterProject Project to bill when accessing "requester pays" buckets. If unset,
 *                         these buckets cannot be accessed.
 *
 **/
public static CloudStorageConfiguration getCloudStorageConfiguration(int maxReopens, String requesterProject) {
    Builder builder = CloudStorageConfiguration.builder()
        // if the channel errors out, re-open up to this many times
        .maxChannelReopens(maxReopens);
    if (!Strings.isNullOrEmpty(requesterProject)) {
        // enable requester pays and indicate who pays
        builder = builder.autoDetectRequesterPays(true).userProject(requesterProject);
    }

    //this causes the gcs filesystem to treat files that end in a / as a directory
    //true is the default but this protects against future changes in behavior
    builder.usePseudoDirectories(true);
    return builder.build();
}
Example #3
Source File: BucketUtils.java    From gatk with BSD 3-Clause "New" or "Revised" License 5 votes vote down vote up 
/**
 * Get an authenticated GCS-backed NIO FileSystem object representing the selected projected and bucket.
 * Credentials are found automatically when running on Compute/App engine, logged into gcloud, or
 * if the GOOGLE_APPLICATION_CREDENTIALS env. variable is set. In that case leave credentials null.
 * Otherwise, you must pass the contents of the service account credentials file.
 * See https://github.com/GoogleCloudPlatform/gcloud-java#authentication
 *
 * Note that most of the time it's enough to just open a file via
 * Files.newInputStream(Paths.get(URI.create( path ))).
 **/
public static java.nio.file.FileSystem getAuthenticatedGcs(String projectId, String bucket, byte[] credentials) throws IOException {
    StorageOptions.Builder builder = StorageOptions.newBuilder()
            .setProjectId(projectId);
    if (null != credentials) {
        builder = builder.setCredentials(GoogleCredentials.fromStream(new ByteArrayInputStream(credentials)));
    }
    // generous timeouts, to avoid tests failing when not warranted.
    StorageOptions storageOptions = setGenerousTimeouts(builder).build();

    // 2. Create GCS filesystem object with those credentials
    return CloudStorageFileSystem.forBucket(bucket, CloudStorageConfiguration.DEFAULT, storageOptions);
}
Example #4
Source File: BucketUtilsUnitTest.java    From gatk with BSD 3-Clause "New" or "Revised" License 5 votes vote down vote up 
@Test
public void testGetCloudStorageConfiguration() {
    String mockProject = "yes";
    int mockReopens = 100;
    CloudStorageConfiguration config = BucketUtils.getCloudStorageConfiguration(mockReopens, mockProject);
    Assert.assertEquals(config.maxChannelReopens(), mockReopens);
    Assert.assertEquals(config.userProject(), mockProject);
}
Example #5
Source File: UseExplicitCredentials.java    From google-cloud-java with Apache License 2.0 5 votes vote down vote up 
public static void main(String... args) throws IOException {
  // Create a file system for the bucket using the service account credentials
  // saved in the file below.
  String myCredentials = "/path/to/my/key.json";
  CloudStorageFileSystem fs =
      CloudStorageFileSystem.forBucket(
          "mybucket",
          CloudStorageConfiguration.DEFAULT,
          StorageOptions.newBuilder()
              .setCredentials(
                  ServiceAccountCredentials.fromStream(new FileInputStream(myCredentials)))
              .build());
  // Can now read and write to the bucket using fs
  // (see e.g. ReadAllLines for an example).
}