Fix exception in GrpcRetryer.

[chronos-tachyon]

What was changed

• Ensure that ChannelManager.getServerCapabilities follows the standard gRPC retry policy, even though it is called by GrpcRetryer.

Why?

In the course of writing new gRPC retry tests for the features repo, it was discovered that any gRPC failures returned by the server in response to WorkflowService.getSystemInfo will cause an uncaught io.grpc.StatusRuntimeException that happens before the actual gRPC retry code gets the chance to run.

Checklist

1. Closes internal JIRA issue SDK-1888. (Sorry, I should have opened it as a GitHub issue and let it sync to JIRA instead, I didn't learn this until after the fact.)

2. How was this tested: using the new features repo tests in Add features tests for gRPC retry behavior features#435 .

[cretz]

After internal discussion, it seems everyone agrees that no when-to-make-first-call behavior should change in the Java SDK (that would be an incompatible change), but of course fixing retry behavior and other similar ones makes sense. Will defer to @Quinn-With-Two-Ns on final approval.

[Quinn-With-Two-Ns]

I don't think the CI failure is a result of any of your changes, I opened a separate issue to address this flake #2038

[Quinn-With-Two-Ns]

As part of the PR can we please also add some unit tests to verify the changes address the issue? I understand the integration tests will live, so maybe just a targeted test of getServerCapabilitiesWithRetryOrThrow since that takes a channel and you can create a in processes server like this:

https://github.com/grpc/grpc-java/blob/master/examples/src/test/java/io/grpc/examples/helloworld/HelloWorldClientTest.java