gke-grpc-gateway-api-with-istio

Steps to deploy gRPC server streaming service running on GKE with istio sidecar and behind Google's Global External Application Load Balancer using the Gateway API, Google-managed certificate for TLS termination at the LB and self-managed certificate for TLS termination at istio.

Prerequisites

These steps expect a GKE cluster with a Gateway Controller ¹ and internet access (to download the prebuilt container image). Also, the usual gcloud CLI configured for your project and kubectl with credentials to your cluster.

You'll also need a working DNS subdomain to point to the load balancer IP.

Sample setup

Setup Steps

• Create a static IP for LB.

  $ gcloud compute addresses create grpc-istio-demo-vip \
    --ip-version=IPV4 \
    --global

• Point the public DNS to the previously created global IP (I'll be using gke-grpc-gateway-api.chimbuc.dns.doit-playground.com)

• Create a Google-managed SSL certificate for LB.

  $ gcloud compute ssl-certificates create gke-grpc-gateway-api-cert \
    --domains=gke-grpc-gateway-api.chimbuc.dns.doit-playground.com \
    --global

• Generate a self-signed certificate for the istio ingressgateway. I am using mkcert ² to generate the certificate.

• TLS is required both between client and GFE, as well as GFE and backend ³. Istio will use the self-signed cert for LB to istio-ingressgateway.

  $ mkcert internal

  Important: The certificate has to use one of the supported signatures compatible with BoringSSL, see ⁴⁵ for more details.

• Install istio with istioctl. The istio-custom.yaml file contains custom configurations required for this setup.

  The default installation creates a L4 load balancer but we do not require the L4 load balancer for this setup. Go to the Istio release page to download the installation file for your OS, or download and extract the latest release automatically (Linux or macOS):

  $ curl -L https://istio.io/downloadIstio | sh -

  Install Istio with the default configuration profile. For example, if the package is istio-1.20.0:

  $ ./istio-1.20.0/bin/istioctl install -f istio-custom.yaml -y

• Create K8S Secret with the self-signed cert.

  $ kubectl create secret tls grpc-istio-demo-tls \
  --cert=internal.pem \
  --key=internal-key.pem \
  --namespace istio-system

• Enable istio sidecar injection for default namespace.

  $ kubectl label namespace default istio-injection=enabled

• Deploy the demo app.

  $ kubectl apply -f manifests/01-demo-app.yaml

  We're using the Cloud Run gRPC Server Streaming sample application⁵ which listens on port 8080.

• Deploy the demo app K8S svc.

  $ kubectl apply -f manifests/02-demo-app-service.yaml

• Deploy gke-l7-global-external-managed gateway.

  $ kubectl apply -f manifests/03-gateway.yaml

• Deploy istio gateway resource for ingressgateway.

  $ kubectl apply -f manifests/04-istio-default-gateway.yaml

• Deploy istio ingressgateway HTTPRoute.

  $ kubectl apply -f manifests/05-istio-default-httproute.yaml

  Don't forget to change the DNS in the manifest.

• Deploy istio ingressgateway HealthCheckPolicy.

  $ kubectl apply -f manifests/06-istio-HealthCheckPolicy.yaml

  Istio ingressgateway will be the only backend added to the load balancer, and istio will perform internal application routing based on the virtual service configuration

• Deploy demo app istio virtual service.

  $ kubectl apply -f manifests/07-demo-app-virtual-service.yaml

• Optional - Deploy Istio telemetry to enable access logs

  $ kubectl apply -f manifests/08-istio-telemetry-for-access-logs.yaml

• Test

  Clone the repository ⁶ and build the client:

  $ git clone https://github.com/GoogleCloudPlatform/golang-samples
  $ cd golang-samples/run/grpc-server-streaming
  $ go build -o cli ./client

  And run the client:

    $ ./cli -server gke-grpc-gateway-api.chimbuc.dns.doit-playground.com:443
    rpc established to timeserver, starting to stream
    received message: current_timestamp: 2023-11-20T13:33:38Z
    received message: current_timestamp: 2023-11-20T13:33:39Z
    received message: current_timestamp: 2023-11-20T13:33:40Z
    received message: current_timestamp: 2023-11-20T13:33:41Z
    received message: current_timestamp: 2023-11-20T13:33:42Z
    received message: current_timestamp: 2023-11-20T13:33:43Z
    received message: current_timestamp: 2023-11-20T13:33:44Z
    received message: current_timestamp: 2023-11-20T13:33:45Z
    received message: current_timestamp: 2023-11-20T13:33:46Z
    received message: current_timestamp: 2023-11-20T13:33:47Z
    end of stream$ 

Footnotes

1. https://cloud.google.com/kubernetes-engine/docs/concepts/gateway-api ↩

2. https://github.com/FiloSottile/mkcert ↩

3. https://cloud.google.com/load-balancing/docs/https#using_grpc_with_your_applications ↩

4. https://github.com/grpc/grpc/issues/6722 ↩

5. https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/kWwLfeIQIBM/9chGZ40TCQAJ ↩ ↩²

6. https://github.com/GoogleCloudPlatform/golang-samples/tree/main/run/grpc-server-streaming ↩