test: introduce new StorageITRunner (#1785)

## New StorageITRunner

Add two new JUnit runner to provide support for Cross Running test in a class across multiple
backends and transports.

When a class is annotated `@CrossRun(transports = {HTTP, GRPC}, backends = {PROD, TEST_BENCH})`
each test defined in the class will be ran four times, once for each pair from the permutation.
Each permutation will add a new entry to the test suite of the form `[Transport][Backend]`.
This name form can be thought of as keys to access a location in an adjacency matrix.

If only a single backed is needed `@SingleBackend()` can be used to denote the backend.

If a class is annotated `@ParallelFriendly` the test suite may attempt to run the tests in
parallel. Currently parallelism factor is set to 2 * core count.

### Field `@Inject`ion

When defining the test several "globally scoped" items are available for injection to the test
class by annotating a public non-final field with `@Inject`.

Instances of Storage, along with BucketInfo, TestBench, etc. are registered as injectable values.
If a field isn't able to be injected an InitializationError will be raised with details as to why.

### `CrossRun.Ignore` & `CrossRun.Exclude`

Not all tests in a class will apply to all intersections, to allow filtering out tests for
these cases there are two new annotations which can be used. If a test is both Ignored and
Excluded, the Ignore will take precedence.

`@Ignore`d functions similar to the standard `@org.junit.Ignore` where the test will be loaded
but not ran.

`@Exclude`d tests will not be loaded into the suite at all, thereby not being reported at all.

### `@Parameterized` running

When a class is annotated `@Parameterized(ParameterProvider)` in addition to and Cross Run
previously resolved, each parameter will result in another dimension added to the suite.
The name of the suite will be extended with another dimension below any cross name,
for example `[HTTP][PROD][myParam]`.

### Runner Registry

A new class `Registry` has been added which is responsible for managing "globally scoped"
instances of resources associated with running integration tests. All injectable instances
are registered with the Registry and will be lazily initialized, reused and cleaned up.

Instances like Storage clients, Backend Buckets, standard objects etc are all managed per
backend.

When a BucketInfo is loaded into the registry it's name is also made available to the
provider for Storage instances. If a test attempts to mutate the state of a bucket the
operation will be interrupted with a vetoing exception causing the test to fail.

## Refactoring Done

1. TestBench is now a Registry Resource and shouldn't be used outside that context. Since
   testbench binds ports it is not a multi instance thing anyway.
2. Remove StorageFixture in favor of Registry Field Injection
3. Remove BucketFixture in favor of Registry Field Injection
4. All IT test now use either of the two new runners
5. Remove ParallelParameterized runner, instead annotate your class with `@ParallelFriendly`
6. Move bucket cleanup utility from BucketFixture to new class BucketCleaner

* test: add annotations for new Integration tests runners

* test: add new Registry subsystem for new Integration Runners

* test: add new Integration Runners

* chore: add toString to ChecksummedTestContent

* test: migrate ITReadMaskTest to use new StorageITParamRunner

Split enclosed classes into their own respective classes ITBlobReadMaskTest and ITBucketReadMaskTest

* test: migrate ITGrpcTest to use new StorageITRunner

* test: migrate ITHmacTest to use new StorageITRunner

* test: migrate ITNotificationTest to use new StorageITRunner

* test: migrate ITKmsTest to use new StorageITRunner

* chore: remove legacy junit fixtures and runner

* test: migrate ITStorageLifecycleTest to use new StorageITRunner

* test: migrate ITRetryConformanceTest to use new StorageITParamRunner

Refactor RetryTestFixture to extend TestWatcher to allow for manual registration

* test: migrate ITBatchTest to use new StorageITRunner

* test: migrate ITServiceAccountTest to use new StorageITRunner

* test: migrate ITSignedUrlTest to use new StorageITRunner

* test: migrate ITOptionRegressionTest to use new StorageITRunner

* test: migrate ITWriteChannelConnectionPoolTest to use new StorageITRunner

* test: migrate ITDownloadBlobWithoutAuth to use new StorageITRunner

* test: migrate ITDownloadToTest to use new StorageITRunner

* test: migrate ITBlobReadChannelTest to use new StorageITRunner

* test: migrate ITBlobWriteChannelTest to use new StorageITRunner

* test: migrate ITObjectChecksumSupportTest to use new StorageITParamRunner

* test: migrate ITObjectTest to use new StorageITRunner

* test: migrate ITBucketTest to use new StorageITRunner

* test: migrate ITBucketLifecycleTest to use new StorageITRunner

* test: migrate ITBucketLifecycleRulesTest to use new StorageITRunner

* test: migrate ITAccessTest to use new StorageITRunner

* chore: consolidate StorageITParamRunner and StorageSuite into StorageITRunner

Author: BenWhitehead

google-cloud-storage/src/main/java/com/google/cloud/storage/GrpcConversions.java

@@ -500,19 +500,18 @@ private Acl bucketAclDecode(com.google.storage.v2.BucketAccessControl from) {
   }
 
   private com.google.storage.v2.BucketAccessControl bucketAclEncode(Acl from) {
-    BucketAccessControl.Builder to =
-        BucketAccessControl.newBuilder()
-            .setEntity(from.getEntity().toString())
-            .setRole(from.getRole().toString())
-            .setId(from.getId());
+    BucketAccessControl.Builder to = BucketAccessControl.newBuilder();
+    ifNonNull(from.getEntity(), entityCodec::encode, to::setEntity);
+    ifNonNull(from.getRole(), Role::toString, to::setRole);
+    ifNonNull(from.getId(), to::setId);
     ifNonNull(from.getEtag(), to::setEtag);
     return to.build();
   }
 
   private Bucket.IamConfig.UniformBucketLevelAccess ublaEncode(BucketInfo.IamConfiguration from) {
     Bucket.IamConfig.UniformBucketLevelAccess.Builder to =
         Bucket.IamConfig.UniformBucketLevelAccess.newBuilder();
-    to.setEnabled(from.isUniformBucketLevelAccessEnabled());
+    ifNonNull(from.isUniformBucketLevelAccessEnabled(), to::setEnabled);
     if (from.isUniformBucketLevelAccessEnabled() == Boolean.TRUE) {
       ifNonNull(
           from.getUniformBucketLevelAccessLockedTimeOffsetDateTime(),

google-cloud-storage/src/main/java/com/google/cloud/storage/TransportCompatibility.java

@@ -33,7 +33,7 @@
 @Target({ElementType.TYPE, ElementType.METHOD, ElementType.FIELD})
 @Documented
 @Inherited
-@interface TransportCompatibility {
+public @interface TransportCompatibility {
 
   Transport[] value();
 

google-cloud-storage/src/test/java/com/google/cloud/storage/BucketFixture.java

@@ -185,8 +185,8 @@ public void readObject(
             () -> StorageClient.create(fakeServer.getInstance().storageSettings()));
 
     @ClassRule(order = 3)
-    public static final StorageFixture storageFixture =
-        StorageFixture.of(() -> fakeServer.getInstance().getGrpcStorageOptions().getService());
+    public static final AutoClosableFixture<Storage> storageFixture =
+        AutoClosableFixture.of(() -> fakeServer.getInstance().getGrpcStorageOptions().getService());
 
     @Test
     public void autoGzipDecompress_true() throws IOException {

google-cloud-storage/src/test/java/com/google/cloud/storage/GzipReadableByteChannelTest.java

@@ -20,11 +20,14 @@
 
 import com.google.cloud.NoCredentials;
 import com.google.cloud.storage.conformance.retry.TestBench;
-import java.util.List;
+import com.google.cloud.storage.it.runner.StorageITRunner;
+import com.google.cloud.storage.it.runner.annotations.Backend;
+import com.google.cloud.storage.it.runner.annotations.Inject;
+import com.google.cloud.storage.it.runner.annotations.SingleBackend;
 import java.util.stream.Collectors;
 import java.util.stream.StreamSupport;
-import org.junit.ClassRule;
 import org.junit.Test;
+import org.junit.runner.RunWith;
 
 /**
  * The interaction of {@link com.google.cloud.ServiceOptions} instance caching has differing
@@ -34,16 +37,17 @@
  * <p>Define some tests to ensue we are correctly integrating with the caching lifecycle
  */
 // Not in com.google.cloud.storage.it because we're testing package local things
+@SuppressWarnings("ResultOfMethodCallIgnored")
+@RunWith(StorageITRunner.class)
+@SingleBackend(Backend.PROD)
 public final class ITStorageLifecycleTest {
-  @ClassRule(order = 1)
-  public static final TestBench TEST_BENCH =
-      TestBench.newBuilder().setContainerName("lifecycle-test").build();
+  @Inject public TestBench testBench;
 
   @Test
   public void grpc() throws Exception {
     GrpcStorageOptions options =
         StorageOptions.grpc()
-            .setHost(TEST_BENCH.getGRPCBaseUri())
+            .setHost(testBench.getGRPCBaseUri())
             .setCredentials(NoCredentials.getInstance())
             .setProjectId("test-project-id")
             .build();
@@ -54,15 +58,9 @@ public void grpc() throws Exception {
     // ensure both instances are the same
     assertThat(service2).isSameInstanceAs(service1);
 
-    // make sure and RPCs can be done
-    List<Bucket> buckets1 =
-        StreamSupport.stream(service1.list().iterateAll().spliterator(), false)
-            .collect(Collectors.toList());
-    assertThat(buckets1).isEmpty();
-    List<Bucket> buckets2 =
-        StreamSupport.stream(service2.list().iterateAll().spliterator(), false)
-            .collect(Collectors.toList());
-    assertThat(buckets2).isEmpty();
+    // make sure an RPC can be done
+    StreamSupport.stream(service1.list().iterateAll().spliterator(), false)
+        .collect(Collectors.toList());
 
     // close the instance
     service1.close();
@@ -71,18 +69,16 @@ public void grpc() throws Exception {
     Storage service3 = options.getService();
 
     assertThat(service3).isNotSameInstanceAs(service1);
-    // make sure and RPCs can be done
-    List<Bucket> buckets3 =
-        StreamSupport.stream(service3.list().iterateAll().spliterator(), false)
-            .collect(Collectors.toList());
-    assertThat(buckets3).isEmpty();
+    // make sure an RPC can be done
+    StreamSupport.stream(service3.list().iterateAll().spliterator(), false)
+        .collect(Collectors.toList());
   }
 
   @Test
   public void http() throws Exception {
     HttpStorageOptions options =
         StorageOptions.http()
-            .setHost(TEST_BENCH.getBaseUri())
+            .setHost(testBench.getBaseUri())
             .setCredentials(NoCredentials.getInstance())
             .setProjectId("test-project-id")
             .build();
@@ -92,26 +88,18 @@ public void http() throws Exception {
 
     // ensure both instances are the same
     assertThat(service2).isSameInstanceAs(service1);
-    // make sure and RPCs can be done
-    List<Bucket> buckets1 =
-        StreamSupport.stream(service1.list().iterateAll().spliterator(), false)
-            .collect(Collectors.toList());
-    assertThat(buckets1).isEmpty();
-    List<Bucket> buckets2 =
-        StreamSupport.stream(service2.list().iterateAll().spliterator(), false)
-            .collect(Collectors.toList());
-    assertThat(buckets2).isEmpty();
+    // make sure an RPC can be done
+    StreamSupport.stream(service1.list().iterateAll().spliterator(), false)
+        .collect(Collectors.toList());
 
     service1.close(); // this should be a no-op for http
 
     // expect the original instance to still be returned
     Storage service3 = options.getService();
 
     assertThat(service3).isSameInstanceAs(service1);
-    // make sure and RPCs can be done
-    List<Bucket> buckets3 =
-        StreamSupport.stream(service3.list().iterateAll().spliterator(), false)
-            .collect(Collectors.toList());
-    assertThat(buckets3).isEmpty();
+    // make sure an RPC can be done
+    StreamSupport.stream(service3.list().iterateAll().spliterator(), false)
+        .collect(Collectors.toList());
   }
 }